Lab 5: JavaScript, Fetching JSON

🎯 Lab Objective

In this lab, you will learn how to fetch JSON from a web server

This lab teaches the following:

1. fetch()
2. JSON data
3. GET and POST

Table of Contents

We are going to use the asynchronous data fetching function fetch(), to get JSON data from a web server.

Fetching is not instant, servers take time to respond. This is why processing is handled with callback functions.

function response_callback( response ) { ... }
function data_callback( data ) { ... }

fetch("www.example.com").then( response_callback ).then( data_callback );

✅ Tip

When passing a callback to a function only use the function’s name (i.e. without the parentheses)

foo( callback ); // correct ✅
foo( callback() ); // incorrect ❌

The data we are going to fetch is a JSON array of emoji.

https://cosc203.cspages.otago.ac.nz/labs/files/emoji.json

📝 Task 1: Fetching JSON

1. Test the above link with a web browser
2. Create a new .js file
3. In a terminal, run: node --version
   • If you’re on version 17 (or higher) skip to (4)
   • If you’re on version 16.x or below you need to install fetch
     • in a terminal, from the same folder as your .js file:
       • npm install -s node-fetch@2
     • add the following to your .js file

       const fetch = require('node-fetch')
       

4. declare the URL as a variable

   let URL = "https://cosc203.cspages.otago.ac.nz/labs/files/emoji.json";
   

5. Write some code to fetch the data

   fetch(URL).then( response_callback ).then( data_callback );
   

6. Before the fetch() define two functions response_callback and data_callback.

   function response_callback( response ) { ... }
   function data_callback( data ) { ... }
   

7. The response_callback processes the server response
   • if response.status != 200 then return;
   • if the status code is 200 return response.text()
8. The data_callback processes the actual data
   • you should first parse the text data into JSON

   let emoji_array = JSON.parse(data);
   

9. Loop over the emoji_array and print all the emoji
10. Run the code in a terminal with: node yourFileName.js

✅ Arrays in JavaScript

let array = [1, 2, 3, 4, 5];

Use square brackets [ ] to access an element

console.log(array[0]); // prints 1

The .length property

console.log(array.length); // prints 5

For loops

for (let i = 0; i < array.length; i++) {
    console.log(array[i]);
}

For each loops

for (let num of array) {
    console.log(num);
}

✅ Visual Studio Code

Use Visual Studio Code’s built-in terminal

Often, we convert JSON between a string format and object format. Strings are useful as they can be stored as files (serialized) and sent across the internet, but objects are easier to work with in code.

To convert a JavaScript object into a string, use JSON.stringify()

const obj = {
    "name": "grinning face",
    "emoji": "😀"
};
const string = JSON.stringify(obj);

To perform the inverse: convert a string into a JavaScript object, use JSON.parse()

const string = '{ "name": "grinning face", "emoji": "😀" },';
const obj = JSON.parse();

The data we fetched is an array, so we can treat it as such.

Each element of the array is a javascript object with 2 name/value pairs.

{
    "name": "smiling face with sunglasses",
    "emoji": "😎"
}

We can access each name/value pair with dot . notation.

let element = all_emoji[70];

console.log( element.name );

console.log( element.emoji );

📝 Task 2: JSON processing

Modify your emoji fetching program to only print the emoji with "smiling" in their name.

✅ Tip

You may find the string method .includes() useful

if ( my_string.includes("smiling") ) { ... }

We are going to improve the chat application from the previous lab by using fetch() to communicate with an Web Service API.

• If you are working in the labs, use this server URL:
  • http://oucs1555.staff.uod.otago.ac.nz:8080
• If you are working from home, you must host the Web Service API
  • download this Git repo
    • https://altitude.otago.ac.nz/cosc203/code/lab05-chat-server
  • follow the instructions in README.md
  • your API server URL will be:
    • http://localhost:8080

📝 Task 3: A Better Chat Application

The provided files are modified from the previous lab.

We will modify lab05.js to fetch messages from a server (via a Web Service API).

1. Download the lab files
   • https://altitude.otago.ac.nz/cosc203/code/lab05-chat-client
2. Test the client app first!
   • in Visual Studio Code start Live Server extension.
   • Open lab05.html
   • Type a user name
   • Type a message, and hit Send
3. Read the code!
   • It is important that you understand these functions:
     • lab05.js > submitNewMessage()
       • new ChatMessage(userName, message, time, "message-self")
     • lab05-utils.js > _create()
     • lab05-utils.js > _newelement()
   • They are refactored from the previous lab.
4. Test the server first!
   • Open the API server with a browser http://oucs1555.staff.uod.otago.ac.nz:8080
     • read about how to interact with the server
   • Test the /messages.json endpoint.
     • http://oucs1555.staff.uod.otago.ac.nz:8080/messages.json
       • If working from home http://localhost:8080/messages.json
     • you should see a JSON array of messages
     • this is the endpoint where we will fetch data from
5. fetch /messages.json in lab05.js > loadAllMessages()
   • Use the pattern fetch(url).then(response_callback).then(data_callback);
     • Response callback should return response.txt();
     • Data callback should:
       • Parse the JSON array JSON.parse(data)
       • For each message in the JSON array
         • call new ChatMessage(sender, message, time, type)
           • sender, message, and time, are all in the JSON object
           • type is a CSS class and should be:
             • "message-self" if userName == message.name
             • "message-other" otherwise.
   • Messages you send will dissapear as they aren’t actually sent to the server (yet)
6. Prevent duplicate messages
   • If you did everything right the chat log will grow infinitely 😢
   • This is because loadAllMessages() is called every 3 seconds (using setTimeout())
   • Modify your data handler callback to wipe all messages before loading new ones
     • remove all existing messages in the chat log container
     • get element <main id="chat-log-container">
       • or: let e = document.getElementById('#id-goes-here');
     • set e.innerHTML = '';
7. Your application should now refresh the chat log every 3 seconds
   • This isn’t the best way to do this.
   • Can you think of a better way?

By default fetch() sends GET requests. In the next task we will send POST requests to post data to the API server.

To send a POST request with fetch() you need to set the right options (the second parameter of fetch).

const fetchOptions = {
    method: "POST",
    body: '{ "name": "data" }', /* stringified JSON */
    headers: { "Content-Type": "application/json" }
}

fetch(URL, fetchOptions).then( post_response_callback ).then( post_data_callback );

Option paramters:

• method is the HTTP request type (GET is default).
• body is the payload, the data we are sending to the server
• headers are connection info and metadata.

📝 Task 4: Create POST requests

Let’s send messages to the server.

Write you code in lab05.js > submitNewMessage()

1. Prepare the data for transmission
   • Format the message data a JSON object
   • it must have 3 name/value pairs ("name", "message", and "time")
     • for time use new Date().getTime();

   const json_object = {
       "name": userName,
       "message": message,
       "time": time
   };
   

2. Use fetch to send the POST request
   • The server’s endpoint for receiving POST requests is: /send-message
   • fetch options:

   const options = {
       method: 'POST',
       body: json_string, // stringify json_object before sending!
       headers: { 'Content-Type': 'application/json' }
   }
   

3. Write 2 callbacks to handle the response and data

   fetch(url, options).then(response_callback).then(data_callback);
   

4. In reponse callback
   • console.log(response.status);:
     • 201 Successfully created
     • 400 Bad Request
   • return response.text();
5. In data callback
   • console.log(data); for debugging
6. Congratulations
   • you can now chat with everyone in the lab!

Improve our chat client so it doesn’t wipe the chat log every 3 seconds.

📝 Task 5: Final Challenge

1. Modify loadAllMessages() to be more efficient.

👉 Click me for hint #1 ↕

Don’t wipe the entire chat history. Instead, only create new messages if they aren’t yet loaded.

👉 Click me for hint #2 ↕

Use message.time to determine if the message has already been loaded.

• smaller value are in the past (already loaded)
• larger values are in the future (not yet loaded)

👉 Click me for hint #3 ↕

Store the most recent (largest) message.time in a global variable.

If a message’s time is smaller the our global variable it’s already on our page.

If a message’s time is larger than our global variable, create a new message.

• Task 1: Fetching JSON
• Task 2: JSON processing
• Task 3: A Better Chat Application
• Task 4: Create POST requests
• Task 5: Final Challenge