Issue with Twitchtv JSON API project: random output of online/offline streams

Hi there,

I have taken a simple approach to the Twitchtv JSON API project: https://codepen.io/bvcx/pen/wjVGRO

I have a function that loops through the stream array and push false to another array if the stream (data.stream) is null, if it’s something other than null it pushes true to the array.

In the next function I loop through the stream array and have an if statement that checks against the online/offline array, if true it appends content reflecting the stream being live, if false it appends content reflecting the stream being offline.

It does work, but not as intended. What streams is online/offline usually changes each time I run the script within a short timeframe (seconds), so something is wrong.

I’m aware that the project needs work, I just want to get basics working first.



        var streamArray = ["ESL_SC2", "OgamingSC2", "cretetion", "freecodecamp", "storbeck", "habathcx", "RobotCaleb", "noobs2ninjas"];
        var onlineOrNotArr = [];

        function offlineOrOnline() {
            for (var i = 0; i < streamArray.length; i++) {

                $.ajax({
                    url: 'https://wind-bow.glitch.me/twitch-api/streams/' + streamArray[i],
                    type: 'get',
                    data: {

                    },

                    dataType: 'json',
                    success: function (data) {

                        if (data.stream === null) {
                            onlineOrNotArr.push(false);
                        } else {
                            onlineOrNotArr.push(true);
                        }


                    }
                });
            }
        }

        function printStreams() {
            var j = 0;
            for (var i = 0; i < streamArray.length; i++) {

                $.ajax({
                    url: 'https://wind-bow.glitch.me/twitch-api/channels/' + streamArray[i],
                    type: 'get',
                    data: {

                    },

                    dataType: 'json',
                    success: function (data) {

                        if (onlineOrNotArr[j++] === true) {
                            $(".streams").append('<div class="results online"><div class="nestedDiv"><img src="' + data.logo + '" height"100px" width="100px"></div>' + '<div class="nestedDiv"><b>' + data.display_name + '</b></div>' + '<div class="nestedDiv">' + data.game + ' : ' + data.status + '</div></div>');
                        } else {
                            $(".streams").append('<div class="results offline"><img src="' + data.logo + '" height"100px" width="100px">' + data.display_name + 'Offline' + '</div>');

                        }

                    }
                });
            }
        }
        offlineOrOnline();
        printStreams();

        function hideOnline() {
            $('.online').css("visibility", "hidden");
            $('.offline').css("visibility", "visible");
        }

        function showOnline() {
            $('.online').css("visibility", "visible");
            $('.offline').css("visibility", "hidden");
        }

        function showAll() {
            $('.online').css("visibility", "visible");
            $('.offline').css("visibility", "visible");
        }

The reason is because jQuery.ajax() is asynchronous, so when you iterate through the list of usernames and make the corresponding API calls, they all go out effectively as fast as the for loop iterates through them without waiting for anything—the responses also come back whenever they can, too.

One way to get around this (the one I’m most familiar with—there are certainly other ways!) is to use Promises. jQuery.ajax() returns a Promise object, which you can use to check whether or not a response has completed before you move on to do something else. Consider the following example:

function apiCall() {
  return $.ajax({
    url: "https://wind-bow.glitch.me/twitch-api/streams/freeCodeCamp",
    type: "get",
    data: {},

    dataType: "json",
    success: function(data) {
      console.log('Data fetched!');
    }
  });
}

apiCall();
console.log('Done fetching!');

The messages logged to the console will always appear in the following order:

"Done fetching!"
"Data fetched!"

Since the object returned by jQuery.ajax() is a Promise object and has the then() method available to it, we can make sure that "Done fetching!" only gets printed in the corrected order as follows:

// Using the same function apiCall() as above. Note that the function
// apiCall() returns $.ajax();

apiCall().then(() => {
    console.log('Done fetching');
});

Output:

"Data fetched!"
"Done fetching!"

To handle multiple apiCall() such as that in your case, you can use Promise.all(); here is a quick example:

Promise.all([apiCall(0), apiCall(1)].then(function(responses) {
    console.log(responses);
    // An array of responses from jQuery.ajax() whose order corresponds to the
    // order in the array you give to Promise.all(), that is the response[0] is the
    // response for apiCall(0) and response[1] is the response for apiCall(1)
});

The rest is up to you! I hope that helps! :smile:

Thanks for the reply on this matter, really appreciate the help.

I may misunderstand Promise, but is it as simple as replace the function calls with the following:

 Promise.all([offlineOrOnline(0), printStreams(1)].then(function(responses) {
            console.log(responses);
           
        }));

I have done this (https://codepen.io/bvcx/pen/wjVGRO), but get an error:

Uncaught TypeError: [offlineOrOnline(...),printStreams(...)].then is not a function

Mm, not quite—the issue is that offlineOrOnline() and printStreams() itself contain a bunch of API calls inside, and they are still going to be sent, and the corresponding results received, at different intervals. The numbers that I used as arguments earlier are also just so that I can refer to them and it’s not an implementation Promise.all().

Perhaps this is a better example:

function apiCall(channel) {
  return $.ajax({
    url: "https://wind-bow.glitch.me/twitch-api/streams/" + channel,
    type: "get",
    data: {},
    dataType: "json",
    success: function(data) {
     //  console.log('Data for ' + channel + ' fetched!', data);
    }
  });
}

const apiCalls = [apiCall('ESL_SC2'), apiCall('freeCodeCamp')];

Promise.all(apiCalls).then(function(responses) {
  console.log(responses); // List of ordered responses from the API

  // Do more things with responses
});


I have deliberately omitted the details for how to produce an array like apiCalls (to get from ['ESL_SC2', 'freeCodeCamp'] to [apiCall('ESL_SC2'), apiCall('freeCodeCamp')]) because I think that’s your task.

Once you have figured that out you will essentially be nesting something similar at the point where I’ve marked Do more things with responses. This isn’t trivial the first time, so keep at it! I highly recommend making a separate Pen to play with the code above first so that you’re not bogged down by irrelevant details, perhaps start with the code below and try to get to the same point as the example above:

function apiCall(channel) {
  return $.ajax({
    url: "https://wind-bow.glitch.me/twitch-api/streams/" + channel,
    type: "get",
    data: {},
    dataType: "json",
    success: function(data) {
     //  console.log('Data for ' + channel + ' fetched!', data);
    }
  });
}

const channels  = ['ESL_SC2', 'freeCodeCamp'];

Good luck!

2 Likes

Thanks for helping out, really appreciate it :slightly_smiling_face:

I got it up and running. Thanks again! The only thing missing before I can start with the finishing touches is producing an array from ['ESL_SC2', 'freeCodeCamp'] to [apiCall('ESL_SC2'), apiCall('freeCodeCamp')]. I’ve been stuck for hours with this, not looking for an answer, but if you could point me in the right direction (i.e documentation, a hint or something) that would have been much appreciated. The closet I got was doing a for loop and pushing everything to an empty apiCalls array, but that gave me a string array: ["apiCall('ESL_SC2')"," apiCall('freeCodeCamp')"].

Current pen: https://codepen.io/bvcx/pen/wjVGRO

Code:

var streamArray = ["ESL_SC2", "OgamingSC2", "cretetion", "freecodecamp", "storbeck", "habathcx", "RobotCaleb", "noobs2ninjas"];
var apiCalls = [apiCall('ESL_SC2'), apiCall('OgamingSC2'), apiCall('cretetion'), apiCall('freecodecamp'), apiCall('storbeck'), apiCall('habathcx'), apiCall('RobotCaleb'), apiCall('noobs2ninjas')];



function apiCall(channel) {
    return $.ajax({
        url: "https://wind-bow.glitch.me/twitch-api/streams/" + channel,
        type: "get",
        data: {},
        dataType: "json",
        success: function (data) {
        }
    });
}


function printStreams(val, channel) {

    $.ajax({
        url: 'https://wind-bow.glitch.me/twitch-api/channels/' + channel,
        type: 'get',
        data: {

        },

        dataType: 'json',
        success: function (data) {
            if (val === null) {
                $(".streams").append('<div class="results offline" onclick="window.open(' + "'" + data.url + "'" + ');"style="cursor:pointer;"><img src="' + data.logo + '" height"100px" width="100px">' + data.display_name + 'Offline' + '</div>');
            } else {
                $(".streams").append('<div class="results online" onclick="window.open(' + "'" + data.url + "'" + ');"style="cursor:pointer;"><div class="nestedDiv"><img src="' + data.logo + '" height"100px" width="100px"></div>' + '<div class="nestedDiv"><b>' + data.display_name + '</b></div>' + '<div class="nestedDiv">' + data.game + ' : ' + data.status + '</div></div>');

            }
        }
    });
}

Promise.all(apiCalls).then(function (responses) { 

    for (var j = 0; j < apiCalls.length; j++) {
        printStreams(responses[j].stream, streamArray[j]);
    }

});


$(document).ready(function(){

    $("#showAll").click(function(){
        $('.online').show();
        $('.offline').show();
    });
    $("#showOffline").click(function(){
        $('.offline').show();
        $('.online').hide();

    });
    $("#showOnline").click(function(){
        $('.online').show();
        $('.offline').hide();
    });
});

Good job on spending the time to try it! But if you think that you are missing something fundamental and just need a hint, don’t wait for hours next time (unless you enjoy the process, which I do sometimes)!

There a few ways to do what we have been talking about, this is an example with Array.prototype.map() (personal preference):

const arr = [0, 1, 2, 3, 4];
const arrIncremented = arr.map(function(m) {
    return m + 1;
});

console.log(arrIncremented); // [1, 2, 3, 4, 5]

function plusOne(num) {
  return num + 1;
}

const arrPlusOne = arr.map(function(m) {
  return plusOne(m);
});

console.log(arrPlusOne); // [1, 2, 3, 4, 5]

Please keep in mind that that a lot of the code below is pseudo code.

The second example is to show how you can map a function into an array. When you map the function plusOne() to each of the elements in arr, you are effectively creating an array like this:

[plusOne(0), plusOne(1), plusOne(2), plusOne(3), plusOne(4)]

They are function calls, so they will get evaluated accordingly to give the output [1, 2, 3, 4, 5].

In the same manner, jQuery.ajax() simply returns and object—it may be weird because of the async nature, but the only difference, really, is that the objects arrive at different times. For example:

function apiCall(channel) {
    // returning the object is important here
    return jQuery.ajax( /* fetch things for the given channel */ );
}

const channels = ['channelA', 'channelB', 'channelC', 'channelD'];
const promises = channels.map(function(channel) {
    return apiCall(channel);
});

Let’s use the same mental model as above and think of the first snapshot of promises as this:

[apiCall('channelA'), apiCall('channelB'), apiCall('channelC'), apiCall('channelD')]

Since the results can arrive at different times, some time shortly after that, and when the first object is returned by jQuery.ajax() (the promise object returned is more complicated than that, it’s simplified down to the details that I think are relevant for the current discussion), it may look like this:

[{ name: 'channelA', info: 'stuffA' }, apiCall('channelB'), apiCall('channelC'), apiCall('channelD')]

Or it may look like this:

[apiCall('channelA'), apiCall('channelB'), { name: 'channelC', info: 'stuffC' }, apiCall('channelD')]

But eventually they all arrive, even though at different times. If we forget about Promise.all() for a moment, imagine what happens when you use a for loop to process the promises array before all API calls have finished:

for (let i = 0, len = promises.length; i < l; i++) {
    const channelObj = promises[i];

    console.log(channelObj.info);
}

The for loop does dutifully go through the code, but since not all API calls have finished at this point, the picture a bit like this:

console.log(apiCall('channelA').info); // Oh, apiCall('channelA') hasn't returned anything yet, next!
console.log(apiCall('channelB').info); // Oh, apiCall('channelB') hasn't returned anything yet, next!
console.log({ name: 'channelC', info: 'stuffC' }.info); // Okay, info of { name: 'channelC', info: 'stuffC' } is 'stuffC', I'll log it to the console
console.log(apiCall('channelD').info); // Oh, apiCall('channelD') hasn't returned anything yet, next!

So, in this case, you see "stuffC" in the console first and, eventually when the other calls finish (in whatever order), the other channel infos come in different order after ‘stuffC’ depending on when their responses of the API calls come back. This is why you have the original problem.

Coming back to Promises.all()—all it does is to make sure that there are no unresolved requests/processes inside the array that you give to it before it does something else (it actually returns a promise object that is similar to jQuery.ajax()). If we apply this to the example above:

const channels = ['channelA', 'channelB', 'channelC', 'channelD'];
const promises = channels.map(function(channel) {
    return apiCall(channel);
});

Promise.all(promises)
    .then(function(data) {
        console.log('Done: ', data);
    });

You could think of the first snapshot as something like this:

Promise.all([apiCall('channelA'), apiCall('channelB'), apiCall('channelC'), apiCall('channelD')])
    .then(function(data) {
        console.log('Done: ', data);
    });

Then some time shortly after (note the comment):

Promise.all([apiCall('channelA'), apiCall('channelB'), { name: 'channelC', info: 'stuffC' }, apiCall('channelD')])
    // Nope not done yet, I need to keep waiting
    .then(function(data) {
        console.log('Done: ', data);
    });

And then some more time later:

Promise.all({ name: 'channelA', info: 'stuffA' }, apiCall('channelB'), { name: 'channelC', info: 'stuffC' }, apiCall('channelD')])
    // Nope still not done yet, I need to keep waiting
    .then(function(data) {
        console.log('Done: ', data);
    });

Eventually:

Promise.all([{ name: 'channelA', info: 'stuffA' }, { name: 'channelB', info: 'stuffB' }, { name: 'channelC', info: 'stuffC' }, { name: 'channelD', info: 'stuffD' }])
     // Looks like it's all done, I shall return a promise object
     // .then() is listening for the promise object—so the callback
     // inside .then() will also run at this point
    .then(function(data) {
        console.log('Done: ', data); // [{ name: 'channelA', info: 'stuffA' }, { name: 'channelB', info: 'stuffB' }, { name: 'channelC', info: 'stuffC' }, { name: 'channelD', info: 'stuffD' }]
    });

As you can see in the array data, the responses order correspond to the input (the array channels), and that’s the reason why Promise.all is useful for situations like this.

I hope that helps! :slight_smile:

1 Like

Thanks a lot for taking the time to reply. This has been immensely informative and helpful :slightly_smiling_face: