Basic Concepts

In this lesson, we learn about the Basic Concepts of the TGrid.

The 1st chapter will handle about the basic theories, what the true Grid Computing being supported by TGrid and what the Remote Function Call are. In the 2nd chapter, we will learn about the basic components. Finally, the last chapter discusses two protocols supported by TGrid: Web Socket and Workers.

If you like practice rather than theory and prefer one code reading rather than hundreds description listenings, just skip this lesson. For those of you, I recommend you to read the next lesson, Learn from Examples, first.

1. Theory

1.1. Grid Computing

Wekipedia says Grid Computing is:

Wikipedia, Grid computing

https://en.wikipedia.org/wiki/Grid_computing

Grid computing is the use of widely distributed computer resources to reach a common goal. A computing grid can be thought of as a distributed system with non-interactive workloads that involve many files. Grid computing is distinguished from conventional high-performance computing systems such as cluster computing in that grid computers have each node set to perform a different task/application. Grid computers also tend to be more heterogeneous and geographically dispersed (thus not physically coupled) than cluster computers. Although a single grid can be dedicated to a particular application, commonly a grid is used for a variety of purposes. Grids are often constructed with general-purpose grid middleware software libraries. Grid sizes can be quite large.

Grids are a form of distributed computing whereby a "super virtual computer" is composed of many networked loosely coupled computers acting together to perform large tasks. For certain applications, distributed or grid computing can be seen as a special type of parallel computing that relies on complete computers (with onboard CPUs, storage, power supplies, network interfaces, etc.) connected to a computer network (private or public) by a conventional network interface, such as Ethernet. This is in contrast to the traditional notion of a supercomputer, which has many processors connected by a local high-speed computer bus.

In the description of Wikipedia about Grid Computing, I want to emphasize the word, "one virtual computer". In my opinion, the true Grid Computing is not just binding multiple computers with network communication and performming common goal. I think that the true Grid Computing means makings computers to be a single virtual computer. A program running on a single computer and another program runninng on the Distributed Processing System with millions of computers, both of them must have similar program code. It's the true Grid Computing.

Do you agree with me?

1.2. Remote Function Call

Grid Computing

I think the real Grid Computing means turning multiple computers into a single virtual computer. Also, code of the program mounted on the virtual computer must be similar with the code running on a single physical computer.

TGrid realizes the true Grid Computing through Remote Function Call. It literally calling remote system's functions are possible. With the Remote Function Call, you can access to objects of remote system as if they have been in my memory from the beginning.

With TGrid and Remote Function Call, it's possible to handle remote system's objects and functions as if they're mine from the beginning. Do you think what that sentence means? Right, being able to call objects and functions of the remote system, it means that current and remote system are integrated into a single virtual computer.

1.3. Demonstration

In the previous chapter, I mentioned that TGrid and Remote Function Call can turn multiple computers into a single virtual computer. Also, the program code of the virtual computer is similar with another program code running on a single physical computer.

Thus, in this section, I will show you some brief demonstration codes. The demonstrations will proof the Remote Function Call can make multiple computers to be a single virtual computer. Also, we need to take a look at how the code using the Remote Function Call.

Hierarchical Composite Single
Hierarchical Composite Single

Three demonstration codes are all come from the 2.2. Learn from Examples lesson. Read the lesson if you want to know more about those demonstration codes. At now, we'll only concentrate one thing. That is, business code of those demonstration codes are all similar.

Whether the program you want to create is built with Grid Computing or runs on a single computer, there is no change in the business logic code. Even if you have two or four computers that make up Grid Computing, there is still no change in your business logic code. Because they are all integrated into one (virtual) computer. How can you make different programs for the same purpose running on a single computer?

I'll conclude this chapter with these words, "This is TGrid. This is Remote Function Call".


import { WorkerConnector } from "tgrid/protocols/workers";
import { Driver } from "tgrid/components";

import { ICompositeCalculator } from "../../controllers/ICalculator";

async function main(): Promise<void>
{
    //----
    // CONNECTION
    //----
    // DO CONNECT
    let connector: WorkerConnector = new WorkerConnector();
    await connector.connect(__dirname + "/calculator.js");

    //----
    // CALL REMOTE FUNCTIONS
    //----
    // GET DRIVER
    let calc: Driver<ICompositeCalculator> = connector.getDriver<ICompositeCalculator>();

    // FUNCTIONS IN THE ROOT SCOPE
    console.log("1 + 6 =", await calc.plus(1, 6));
    console.log("7 * 2 =", await calc.multiplies(7, 2));

    // FUNCTIONS IN AN OBJECT (SCIENTIFIC)
    console.log("3 ^ 4 =", await calc.scientific.pow(3, 4));
    console.log("log (2, 32) =", await calc.scientific.log(2, 32));

    try
    {
        // TO CATCH EXCEPTION IS STILL POSSIBLE
        await calc.scientific.sqrt(-4);
    }
    catch (err)
    {
        console.log("SQRT (-4) -> Error:", err.message);
    }

    // FUNCTIONS IN AN OBJECT (STATISTICS)
    console.log("Mean (1, 2, 3, 4) =", await calc.statistics.mean(1, 2, 3, 4));
    console.log("Stdev. (1, 2, 3, 4) =", await calc.statistics.stdev(1, 2, 3, 4));

    //----
    // TERMINATE
    //----
    await connector.close();
}
main().catch(exp =>
{
    console.log(exp);
});

Hierarchical


import { WorkerConnector } from "tgrid/protocols/workers";
import { Driver } from "tgrid/components";

import { ICompositeCalculator } from "../../controllers/ICalculator";

async function main(): Promise<void>
{
    //----
    // CONNECTION
    //----
    // DO CONNECT
    let connector: WorkerConnector = new WorkerConnector();
    await connector.connect(__dirname + "/calculator.js");

    //----
    // CALL REMOTE FUNCTIONS
    //----
    // GET DRIVER
    let calc: Driver<ICompositeCalculator> = connector.getDriver<ICompositeCalculator>();

    // FUNCTIONS IN THE ROOT SCOPE
    console.log("1 + 6 =", await calc.plus(1, 6));
    console.log("7 * 2 =", await calc.multiplies(7, 2));

    // FUNCTIONS IN AN OBJECT (SCIENTIFIC)
    console.log("3 ^ 4 =", await calc.scientific.pow(3, 4));
    console.log("log (2, 32) =", await calc.scientific.log(2, 32));

    try
    {
        // TO CATCH EXCEPTION IS STILL POSSIBLE
        await calc.scientific.sqrt(-4);
    }
    catch (err)
    {
        console.log("SQRT (-4) -> Error:", err.message);
    }

    // FUNCTIONS IN AN OBJECT (STATISTICS)
    console.log("Mean (1, 2, 3, 4) =", await calc.statistics.mean(1, 2, 3, 4));
    console.log("Stdev. (1, 2, 3, 4) =", await calc.statistics.stdev(1, 2, 3, 4));

    //----
    // TERMINATE
    //----
    await connector.close();
}
main().catch(exp =>
{
    console.log(exp);
});

Composite

import { WebConnector } from "tgrid/protocols/web";
import { Driver } from "tgrid/components";

import { ICompositeCalculator } from "../../controllers/ICalculator";

async function main(): Promise<void>
{
    //----
    // CONNECTION
    //----
    let connector: WebConnector = new WebConnector();
    await connector.connect("ws://127.0.0.1:10102");

    //----
    // CALL FUNCTIONS
    //----
    // GET DRIVER

    let calc: Driver<ICompositeCalculator> = connector.getDriver<ICompositeCalculator>();
    // FUNCTIONS IN THE ROOT SCOPE
    console.log("1 + 6 =", await calc.plus(1, 6));
    console.log("7 * 2 =", await calc.multiplies(7, 2));

    // FUNCTIONS IN AN OBJECT (SCIENTIFIC)
    console.log("3 ^ 4 =", await calc.scientific.pow(3, 4));
    console.log("log (2, 32) =", await calc.scientific.log(2, 32));

    try
    {
        // TO CATCH EXCEPTION IS STILL POSSIBLE
        await calc.scientific.sqrt(-4);
    }
    catch (err)
    {
        console.log("SQRT (-4) -> Error:", err.message);
    }

    // FUNCTIONS IN AN OBJECT (STATISTICS)
    console.log("Mean (1, 2, 3, 4) =", await calc.statistics.mean(1, 2, 3, 4));
    console.log("Stdev. (1, 2, 3, 4) =", await calc.statistics.stdev(1, 2, 3, 4));

    //----
    // TERMINATE
    //----
    await connector.close();
}
main();

Single

import { CompositeCalculator } from "../../providers/Calculator";

function main(): void
{
    //----
    // CALL FUNCTIONS
    //----
    // CONSTRUCT CALCULATOR
    let calc: CompositeCalculator = new CompositeCalculator();

    // FUNCTIONS IN THE ROOT SCOPE
    console.log("1 + 6 =", calc.plus(1, 6));
    console.log("7 * 2 =", calc.multiplies(7, 2));

    // FUNCTIONS IN AN OBJECT (SCIENTIFIC)
    console.log("3 ^ 4 =", calc.scientific.pow(3, 4));
    console.log("log (2, 32) =", calc.scientific.log(2, 32));

    try
    {
        // TO CATCH EXCEPTION
        calc.scientific.sqrt(-4);
    }
    catch (err)
    {
        console.log("SQRT (-4) -> Error:", err.message);
    }

    // FUNCTIONS IN AN OBJECT (STATISTICS)
    console.log("Mean (1, 2, 3, 4) =", calc.statistics.mean(1, 2, 3, 4));
    console.log("Stdev. (1, 2, 3, 4) =", calc.statistics.stdev(1, 2, 3, 4));
}
main();

1 + 6 = 7
7 * 2 = 14
3 ^ 4 = 81
log (2, 32) = 5
SQRT (-4) -> Error: Negative value on sqaure.
Mean (1, 2, 3, 4) = 2.5
Stdev. (1, 2, 3, 4) = 1.118033988749895

2. Components

Sequence Diagram

// ACCESS FROM NAMESPACES
import tgrid = require("tgrid");

let communicator: tgrid.components.Communicator;
let driver: tgrid.components.Driver<Controller>;

// IMPORT FROM MODULE
import { Communicator, Driver } from "tgrid/components";

// IMPORT FROM FILES
import { Communicator } from "tgrid/components/Communicator";
import { Driver } from "tgrid/components/Driver";

2.1. Communicator

Communicates with Remote System.

Communicator is a class taking full responsibility to network communication with remote system. You can register a Provider, an object would be provided to remote system, thorugh this Communicator. Also, Driver<Controller>, who can use remote system's Provider, is created by this Communicator.

Communicator.getDriver<Controller>

In addition, Communicator is the top-level abstract class. All the classes responsible for network communication in TGrid inherit this Communicator class. If you want to know more, go to the 3. Protocols chapter.

import { WebServer, WebAcceptor } from "tgrid/protocols/web";
import { CompositeCalculator } from "../../providers/Calculator";

async function main(): Promise<void>
{
    let server: WebServer = new WebServer();
    await server.open(10102, async (acceptor: WebAcceptor) =>
    {
        await acceptor.accept(new CompositeCalculator());
    });
}
main();

Server

import { WebServer, WebAcceptor } from "tgrid/protocols/web";
import { CompositeCalculator } from "../../providers/Calculator";

async function main(): Promise<void>
{
    let server: WebServer = new WebServer();
    await server.open(10102, async (acceptor: WebAcceptor) =>
    {
        await acceptor.accept(new CompositeCalculator());
    });
}
main();

Client

import { WebConnector } from "tgrid/protocols/web";
import { Driver } from "tgrid/components";

import { ICompositeCalculator } from "../../controllers/ICalculator";

async function main(): Promise<void>
{
    //----
    // CONNECTION
    //----
    let connector: WebConnector = new WebConnector();
    await connector.connect("ws://127.0.0.1:10102");

    //----
    // CALL FUNCTIONS
    //----
    // GET DRIVER

    let calc: Driver<ICompositeCalculator> = connector.getDriver<ICompositeCalculator>();
    // FUNCTIONS IN THE ROOT SCOPE
    console.log("1 + 6 =", await calc.plus(1, 6));
    console.log("7 * 2 =", await calc.multiplies(7, 2));

    // FUNCTIONS IN AN OBJECT (SCIENTIFIC)
    console.log("3 ^ 4 =", await calc.scientific.pow(3, 4));
    console.log("log (2, 32) =", await calc.scientific.log(2, 32));

    try
    {
        // TO CATCH EXCEPTION IS STILL POSSIBLE
        await calc.scientific.sqrt(-4);
    }
    catch (err)
    {
        console.log("SQRT (-4) -> Error:", err.message);
    }

    // FUNCTIONS IN AN OBJECT (STATISTICS)
    console.log("Mean (1, 2, 3, 4) =", await calc.statistics.mean(1, 2, 3, 4));
    console.log("Stdev. (1, 2, 3, 4) =", await calc.statistics.stdev(1, 2, 3, 4));

    //----
    // TERMINATE
    //----
    await connector.close();
}
main();

2.2. Provider

Object being provided for Remote System

Provide is an object provided for the remote system. The other system can remotely call the Provider's functions through Driver<Controller>.

export class Calculator
{
    public plus(x: number, y: number): number;
    public minus(x: number, y: number): number;
    public multiplies(x: number, y: number): number;
    public divides(x: number, y: number): number;

    public scientific: Scientific;
    public statistics: Statistics;
}

2.3. Controller

Interface of Provider

Controller is an interface of Provider, provided from the remote system.

export interface ICalculator
{
    plus(x: number, y: number): number;
    minus(x: number, y: number): number;
    multiplies(x: number, y: number): number;
    divides(x: number, y: number): number;

    scientific: IScientific;
    statistics: IStatistics;
}

2.4. Driver

Driver of Controller for Remote Function Call

Driver is an object for calling functions of the remote system. It is designed to specify a generic parameter Controller. Through the Driver<Controller>, you can call functions of the Provider, provided from the remote system. In other words, Remote Function Call means to call functions of Provider, provided from the remote system, through the Driver<Controller>.

Also, when you assign a Controller into the generic parameter of Driver, the return type of all functions defined in the Controller would be promisified. If an internal object exists in the target Controller, the object type would be converted as Driver<object>. Since those conversions are recursviely repeated, all the functions defined in the Controller would be promisified, regardless of their hierarchy depth.

type Driver<ICalculator> = 
{
    plus(x: number, y: number): Promise<number>;
    minus(x: number, y: number): Promise<number>;
    multiplies(x: number, y: number): Promise<number>;
    divides(x: number, y: number): Promise<number>;

    readonly scientific: Driver<IScientific>;
    readonly statistics: Driver<IStatistics>;
};

Driver ignores atomic variables

In Driver<Controller>, all of the atomic variables like number and string are being ignored.

As you can see from the below code, all of the atomic variables defined in the Controller are disappeared. Therefore, when designing a Provider and you want to provide atomic variables to the remote system, you should define a getter or setter method like below.

  • Something.getValue()
  • Something.setValue()
interface ISomething
{
    getValue(): number;
    setValue(val: number): void;

    value: number;
}

type Driver<ISomething> = 
{
    getValue(): Promise<number>;
    setValue(val: number): Promise<void>;
};

3. Protocols

3.1. Web Socket

// ACCESS FROM NAMESPACES
import tgrid = require("tgrid");

let server: tgrid.protocols.web.WebServer;
let acceptor: tgrid.protocols.web.WebAcceptor;

// IMPORT FROM MODULE
import { WebServer, WebAcceptor } from "tgrid/protocols/web";

// IMPORT FROM FILES
import { WebServer } from "tgrid/protocols/web/Communicator";
import { WebAcceptor } from "tgrid/protocols/web/Driver";

3.1.1. Outline

TGrid supports Web Socket protocol.

3.1.2. Tutorials

TGrid provides detailed API Documents for protocols.web module. Also, example codes and demo projects using Web Socket protocol are already prepared. I recommend you to utilze not only API Documents but also below example codes and demo projects.

3.1.3. Module

Class Web Browser NodeJS Usage
WebServer X O Opens a Web Socket Server
WebAcceptor X O RFC communication with Client
WebConnector O O RFC communication with Server

There are only thress classes in the protocols.web module. The first is WebServer is a class, which is designed for opening a Web Socket server. The second is WebAcceptor class, which is created whenever a client connects to the WebServer and taking responsibility to RFC communication with the client. The last is WebConnector class, which is designed for Web Socket client.

Between those classes, Communicator classes are WebAcceptor and WebConnector. Of course, WebServer can certianly open a WebSocket server and creates WebAcceptor objects whenever client connects, however, it's never the Communicator class using the RFC.

3.2. Workers

// ACCESS FROM NAMESPACES
import tgrid = require("tgrid");

let server: tgrid.protocols.workers.WorkerServer;
let connector: tgrid.protocols.workers.WorkerConnector;

// IMPORT FROM MODULE
import { WorkerServer, WorkerConnector } from "tgrid/protocols/workers";

// IMPORT FROM FILES
import { WorkerServer } from "tgrid/protocols/workers/WorkerServer";
import { WorkerConnector } from "tgrid/protocols/workers/WorkerConnector";

3.2.1. Outline

TGrid supports Worker and SharedWorker protocols.

Why workers be categorized into network system

Worker like Network

Worker is invented for supporting multi-threading in the web browsers. However, unlike threads in normal programming languages, workers cannot share memory variables. As workers cannot share memory variables, interactions with web browser and worker (or with worker and worker) are implemented by [MessageChannel] which utilizes binary data communication.

Interactions are done by not sharing memory variables but binary communcation? Isn't this where you heard a lot? That's right, it's the most typical form of distributed processing system using network communication. In other words, MessageChannel used by workers is same with the network communication, in the conceptual level.

Let's review the workers. Workers are physically created in the thread level. However, in conceptual level and considering how they interact, workers are a type of network system. In such reason, TGrid interpretes Worker and SharedWorker as network protocols and supports Remote Function Call for those protocols.

3.2.2. Tutorials

TGrid provides detailed API Documents for protocols.workers module. Also, example codes and demo projects using Worker protocol are already prepared. I recommend you to utilze not only API Documents but also below example codes and demo projects.

3.2.3. Module

Class Web Browser NodeJS Usage
WorkerConnector O O Creates Worker and RFC communication
WorkerServer O O Worker itself. RFC communication with client
SharedWorkerConnector O X Create or connect to SharedWorker and RFC communication
SharedWorkerServer O X SharedWorker itself. Creates server
SharedWorkerAcceptor O X RFC communication with client

Classes in the protocols.workers module can be divided into two categories. The first is Worker and the other is SharedWorker. The key differences of two categories are their cardinalities between server and clients; Worker is 1: 1 and SharedWorker is 1: N.

WorkerConnector creates a Worker instance and the Worker instances opens WorkerServer. After the instance creation, WorkerConnector connects to the WorkerServer directly and communicates with Remote Function Call. Note that, the WorkerServer can accept only a connection, a WorkerConnector who created the Worker instance.

It is a bit unusual for the SharedWorkerConnector. SharedWorkerConnector create a new SharedWorker instance before connection if target file has not mounted yet, otherwise just connects to ordinary SharedWorker instance if target file has already been mounted. By such characteristics, SharedWorkerServer can accept multiple connections like WebServer. Of course, Communicator classes, reponsible for RFC communication, of the SharedWorker are SharedWorkerConnector and SharedWorkerConnector.

SharedWorker for NodeJS

SharedWorker is a technology that only web browsers support, and NodeJS cannot use it.

However, SharedWorker can be very powerful weapon. In my opinion, SharedWorker can be a great solution for simulating the Web Socket server. Also, developing NodeJS program, I often desire the SharedWorker to develop common background process.

So I'm planning to build a polyfill library that can use SharedWorker in the NodeJS. However, I still don't know how to implement it. If anyone reading this article knows how to build the polyfill library, please help me.

results matching ""

    No results matching ""