Lab 5: JavaScript, Fetching and Processing JSON
π― Lab ObjectiveIn this lab, you will learn how to fetch JSON from a web server and process it.
This lab teaches the following:
- fetch()
- Processing JSON data
- GET and POST
Table of Contents
For the first part of this lab you should use a new project.
Create a new folder named lab05-fetch
and create the files in that folder.
We are going to use the asynchronous data fetching function fetch()
to get JSON data from a web server.
Fetching is not instant, servers take time to respond. This is why processing is handled with callback functions — the callback function will be called once the response arrives. Even after the response arrives, the associated data might not be finished downloading, so there is often another callback function for once download has been completed.
function responseCallback( response ) { ... }
function dataCallback( data ) { ... }
fetch("www.example.com").then( responseCallback ).then( dataCallback );
β TipReminder from the last lab — when passing a callback to a function only use the function’s name (i.e. without the parentheses)
foo( callback ); // correct β foo( callback() ); // incorrect β
The data we are going to fetch is a JSON array of emoji. The following is the URL for the JSON data file:
https://cosc203.cspages.otago.ac.nz/labs/files/emoji.jsonπ Task 1: Fetching JSON
Test the above link with a web browser. You should see some information about a bunch of emojis appear in JSON format.
Create a new
.js
file in your project. This file is going to be used to fetch the emoji data for the chat app, so name the file appropriately.
In the JS file, declare the JSON URL as a constant:
const URL = "https://cosc203.cspages.otago.ac.nz/labs/files/emoji.json";
Add the following three functions to your JS file:
// callback for when the fetch response arrives function responseCallback(response) { } // callback for when the data is ready to use function dataReadyCallback(data) { } // callback for when a fetch error occurs function fetchErrorCallback(error) { }
Add the fetch call to retrieve the JSON. This goes at the bottom of the JS file, at the root level.
fetch(URL) .then(responseCallback) .then(dataReadyCallback) .catch(fetchErrorCallback);
The
responseCallback
processes the server response.
- if
response.status != 200
thenreturn;
since something has gone wrong. The catch block will most likely be triggered in this case, so you don’t need any error handling code here.- if the status code is 200 then return the JSON text using:
return response.text();
The
fetchErrorCallback
is used to notify the user when the fetch fails, so add the following to that function:alert("An error occured when fetching the emoji data. Please try again later."); console.log(error);
The
dataCallback
processes the actual data.
- you should first parse the JSON into JavaScript objects using:
let emojis = JSON.parse(data);
This will produce an array containing an object for each emoji.
Loop over the
emojis
array (see below if you don’t remember how to do a for-each loop in JavaScript) and print all the emojis usingconsole.log
.To test the code, you have two options:
Run the code using Node.js in a terminal with:
node yourFileName.js
This assumes that you have Node.js installed (which is installed on the lab machines).
Create a simple HTML file and link the JS file via the script tag. You can then load the HTML file in the browser using the Live Server and look in the browser dev tools console for the output.
β Arrays in JavaScriptlet array = [1, 2, 3, 4, 5];
Use square brackets
[ ]
to access an elementconsole.log(array[0]); // prints 1
The
.length
propertyconsole.log(array.length); // prints 5
For loops
for (let i = 0; i < array.length; i++) { console.log(array[i]); }
For each loops
for (let element of array) { console.log(element); }
β Visual Studio CodeUse Visual Studio Code’s built-in terminal
Often, we convert JSON between a string
format and object
format. Strings are useful as they can be stored as files (serialized) and sent across the internet, or stored directly in the browser’s localStorage, but objects are easier to work with in code.
To convert a JavaScript object into a string
, use JSON.stringify()
const obj = {
"name": "grinning face",
"emoji": "π"
};
const string = JSON.stringify(obj);
To perform the inverse: convert a string
into a JavaScript object
, use JSON.parse()
const string = '{ "name": "grinning face", "emoji": "π" }';
const obj = JSON.parse(string);
In case you are wondering what JSON stands for — JavaScript Object Notation. It is the object literal format of JavaScript for creating objects. So, you can write JSON directly in a JavaScript file to create an object:
const grin = { "name": "grinning face", "emoji": "π" }
console.log(grin.emoji);
So, why do we need to use JSON.parse()
— why don’t we just assign the JSON directly to a variable? Security. We can’t trust the JSON coming from a remote location to not include malicious functions. If we assigned it directly to a variable then those malicious functions would be included in our system. The JSON.parse()
function will strip out any functions to ensure that what we end up with is safe to use in our application.
The data we fetched is an array, so we can treat it as such.
Each element of the array is a normal JavaScript object with two data fields:
{
"name": "smiling face with sunglasses",
"emoji": "π"
}
We can access each field with usual dot .
notation for accessing the contents of an object:
let element = emojis[70];
console.log( element.name );
console.log( element.emoji );
π Task 2: JSON processingModify your emoji fetching program to only print the emoji with"smiling"
in their name.
β TipYou can do partial string matching in JavaScript with the string
includes()
method to search for substrings:if ( myString.includes("smiling") ) { ... }
To make the partial search case-insensitive, you can convert both strings to lowercase first:
if ( myString.toLowerCase().includes(searchString.toLowerCase()) ){ ... }
You may find this tip useful in the assignment 1.
We are going to improve the chat application from the previous lab by using fetch()
to communicate with an Web Service API.
Rather than chatting with Polly we will use a web service that allows you to chat with other people (or yourself using multiple browser tabs).
If you are working in the labs (using the lab machines or your own laptop), use this server URL:
This will allow you to chat with the rest of the class.
If you are working from home, you must host the web service yourself.
You will need Node.js installed, so if you don’t have that, download and install from:
clone the following Git repo:
follow the instructions in
README.md
your server URL will be:
π Task 3: A Better Chat ApplicationThe provided files are modified from the previous lab.
We will modify
lab05.js
to fetch messages from a server (via a Web Service API).
Download or clone the lab files from the following repository:
Choose the correct URL for the web service at the top of
lab05.js
. If you are in the labs, or otherwise using the University network then the first URL is fine. If you are working from home, or outside of the University then use the second URL.Test the client app first!
- in
Visual Studio Code
startLive Server
extension.- Open
lab05.html
- Type a user name
- Type a message, and hit Send
Read the code!
- It is important that you understand these functions:
lab05.js
>submitNewMessage()
new ChatMessage(userName, message, time, "message-self")
lab05-utils.js
>_create()
lab05-utils.js
>_newelement()
- They are refactored versions of the code that you saw in the previous lab.
Test the server first!
- Open the API server with a browser http://snafu.staff.uod.otago.ac.nz:8070
- read about how to interact with the server
- Test the
/messages.json
endpoint.
- http://snafu.staff.uod.otago.ac.nz:8070/messages.json
- If working from home and using the self-hosted web service, then look at http://localhost:8070/messages.json
- you should see a JSON array of messages
- this is the endpoint where we will fetch data from
Add code to
lab05.js
>loadAllMessages()
to fetch/messages.json
- Use the pattern:
fetch(url) .then(responseCallback) .then(dataCallback) .catch(errorCallback);
- The
responseCallback
function shouldreturn response.text();
- The
dataCallback
function should:
- Parse the JSON
JSON.parse(data)
and store the resulting array in a variable.- For each message object in the array:
- call
new ChatMessage(name, message, time, type)
- name, message, and time, are all in the message object
- type is a CSS class and should be:
"message-self"
ifuserName === message.name
"message-other"
otherwise.- Messages you send will scroll off the top of the page since they aren’t actually being sent to the server (yet).
Prevent duplicate messages
- If you did everything right the chat log will grow infinitely π’
- This is because
loadAllMessages()
is called every 3 seconds (usingsetTimeout()
)- Modify your
dataCallback
function to wipe all messages before loading new ones
- There is a
clearMessages()
function inlab05-utils.js
that will clear the old messages.- Call that function before creating the new messages.
Your application should now refresh the chat log every 3 seconds
- This isn’t the best way to do this.
- Can you think of a better way?
By default fetch()
sends GET requests. In the next task we will send POST requests to post your messages to the chat web service.
To send a POST request with fetch()
you need to set the right options in the payload (the second parameter of fetch).
const fetchPayload = {
method: "POST",
body: '{ "name": "data" }', /* stringified JSON */
headers: { "Content-Type": "application/json" }
}
fetch(API_URL, fetchPayload)
.then(postResponseCallback)
.then(postDataCallback)
.catch (postErrorCallback);
Payload parameters:
method
is the HTTP request type (GET
is default).body
is the payload, the data we are sending to the serverheaders
are connection info and metadata.
π Task 4: Create POST requestsLet’s send your messages to the service.
Write your code in
lab05.js
>submitNewMessage()
Prepare the data for transmission
- Format the message data into an object that can be passed as message object
- it must have 3 data fields (
"name"
,"message"
, and"time"
)
- for time use
new Date().getTime()
const messageObj = { "name": userName, "message": message, "time": time };
Use fetch to send the
POST
request
- The server’s endpoint for receiving POST requests is:
/send-message
- fetch options:
const postPayload = { method: 'POST', body: messageJSON, // stringify messageObj before sending! headers: { 'Content-Type': 'application/json' } }
Write 3 callback functions to handle the response, data, and error callbacks.
Send the request using
fetch(API_URL + "/send-message", postPayload) .then(postResponseCallback) .then(postDataCallback) .catch(postErrorCallback);
In the
postResponseCallback(response)
function
console.log(response.status);
:
201
means that the message was successfully POSTed400
means that what you sent was not correctreturn response.text();
In the
postDataCallback(data)
function
console.log(data);
for debuggingIn the
postErrorCallback(error)
function
console.log(error)
for debuggingCongratulations
- you can now chat with everyone in the lab (assuming you are using the shared chat service)!
Improve our chat client so it doesn’t wipe the chat log every 3 seconds.
π Task 5: Optional Challenge
- Modify
loadAllMessages()
to be more efficient.
This lab is worth marks. be sure to get signed off.