GSoC '22 Report

Liquid Galaxy Space Chess

Mentors

Víctor Sánchez & Andreu Ibañez

Introduction

The Google summer of code program has been one of the greatest experiences of my life. During the summer I developed the first chess application in the world that uses satellite communications.
The project idea was to create a space-related visualization project in collaboration with Hydra Space & Liquid Galaxy. The target is to use the Liquid Galaxy cluster to visualize a collaborative chess game. In this game humans on Earth play against a satellite AI.

What is Liquid Galaxy? -

Liquid Galaxy is an open-source project founded by Google.
Liquid Galaxy started out as a panoramic multi-display Google Earth viewer but has evolved to become a general data visualization tool for operations, marketing, and research.

Liquid Galaxy gives the ability to fly around Google Earth, view panoramic video and photos, develop interactive tours, and graphically display geographic information data, but it is also a nice plataform for multipurpose visualizations.

What is Hydra-Space? -

Hydra Space is an engineering company based in Madrid, Spain dedicated to satellite IoT communications with impressive full inhouse technology. Hydra Space is a member of "Madrid Innovation and New Space Cluster" (MINSC)

What is Hybridium? -

Hybridium provides an all-in-one hybrid auditorium event technology designed to deliver the most impacful events.

The solution features:
Large scale video-wall and hologram technologies, uniting participants for exceptionally immersive and engaging hybrid collaboration, training and events.

Hybridium makes hybrid and distance learning a truly immersive, collaborative and engaging experience, enabling organizations to extend their geographical reach. Frictionless real-time high-quality video, chat and content sharing allows participants to collaborate as if they were in the same physical space.

LG Space Chess


LG Space chess is the newest way of playing chess.
Remote chess is played in various forms, traditionally through the postal system or its updated version (email), less common methods are fax, homing pigeon, and phone. Today it is usually played throughout a dedicated server. Now, Liquid Galaxy has created the first chess in the world capable of playing against a satellite.

The main idea is that the players vote for a specific movement. The most voted movement will be selected as Earth's choice to play against the satellite.

To achieve this we have set a communication protocol so we can exchange data between the ground-station and the satellite.
Firstly, we must understand the structure of the project.

• Satellite with a chess engine
• Ground station that communicates with the satellite
• Cloud function that communicates with the ground station
• Web controller that communicates with the screens
• Screen visualization

The basic workflow is the following:

- First Case:
• The satellite sends the chessboard status to the ground station and the movement selected by the satellite's AI.
• The cloud function fetch the packet from the ground station and update the controller's database with the packet content.

- Second Case:
• The cloud functions get the most voted move, create the package and send it to the ground-station.
• The ground station sends the package to the satellite.
• The satellite processes the package and updates its chess engine with the Earth's move.

workflow

LG Space chess also has two more game modes, Local Play where the user can play against the local machine, and a “Top Games” mode where the user can reproduce chess master's games.


What work has been done? - Description


LG Space Chess was born in 2022 coded by Pablo Sanchidrián.
LG Space chess as explained before is divided into three subprojects:
• Cloud Functions
• Web/Android Controller
• Screen visualization for the Liquid Galaxy Cluster

• Cloud Functions
These cloud functions run every certain amount of time and are responsible for the connection with the ground station and keep the database updated.

There are the two cloud functions:
- The first one fetches the package the satellite left in the ground station. If there is a package, take it, processes it, and update the database with its content. If there is no package the function aborts the operation.

- The second one fetches all the players' votes and takes the most voted. With the most voted move we create a package that will be sent to the ground station if there are no votes, the cloud function will vote automatically instead of the players using a chess engine.
• Controller
The controller will be the app that the users will connect to play against the satellite, against the local machine or play back chess master's games, as well as get the current location of the satellite (map visualization).
• Screen visualization
3D visualization of the current chess play. Every user with the controller will be able to connect to the screens. Instantly its chessboard status will be displayed on the screens, and every single move or change they make will be reflected on the screens. The user can also view an illustration of how this game works, where he can see the exchange of data between the ground station and the satellite.


What work has been done? - Project Goals and Code


  • Cloud Functions [DONE]
    • Connection with the ground station, SFTP - Example Code
    • Create "sendStatus" function - Code
    • Create player's move packet - Code
    • Create "getStatus" function - Code
    • Create binary to FEN converter function - Code
  • Controller [DONE]
    • Firebase Auth - Code
    • Voting system - Code
    • "Satellite" game-mode - Code
    • "Local Play" game-mode - Code
    • "Top play" reproducer - Code
    • Get satellite's latitude & longitude - Code
    • View satellite location endpoint - Code
    • LGSetting (buttons: Connect/Disconnect, Reboot, Relaunch, Poweroff) - Code
    • Connection management functions - Code
  • Screen Visualization [DONE]
    • Chessboard 3D visualization - Code
    • Chessboard FEN Load function - Code
    • Chessboard Move Logic - Code
    • Game Illustration - Code
    • Camera move animation function - Example Code


LG Space Chess by Pablo Sanchidrián.
Designed and fully developed in 2022.
Every single contribution is documented in the source code.

Full Project Documentation - Link

Project Screenshots

Controller dashboard page [Connected]

dashboard

Find Satellite Page

find satellite

LG Settings Modal

lg setting modal

Screens Chessboard view

chessboard view

Screens Game Illustration

illustration

3D rendering template code for the Liquid Galaxy Cluster


I have developed a code template for other potential current and future contributors. Here you will find the boilerplate code to render any other 3D visualization/game on the Liquid Galaxy Screens.

The template contains:

  • Server Initialization (Server + SocketIO)
  • Server screen's setup (Generate super-resolution)
  • Screen's setup (Visualization setup)
  • Working example with mouse events

Explanation

In order to create the visualization, we need to set/follow a screen distribution. We have used the following:

screen order
As you can see, the center screen is the master (1). On the left, you find the odd screens; on the right, you find the even screens. So the basic schema is: (...7, 5, 3, MASTER, 2, 4, 6...)

• Server Side
What we need to do first is to create an express application. With use we tell the express application to use the cors module. We will need this if we want to connect from another page not hosted by this server.

The second step is to create a Socket-HTTP server that starts with the express app config, and again we treat the cors and more config related to it.


var app = express();
app.use(cors());
var http = httpImport.createServer(app);

var io = new Server(http, {
    cors: {
        origin: '*',
        methods: ['GET', 'POST'],
    },
    handlePreflightRequest: (req, res) => {
        const headers = {
            'Access-Control-Allow-Headers': 'Content-Type, Authorization',
            'Access-Control-Allow-Origin': req.headers.origin,
            'Access-Control-Allow-Credentials': true,
        };
        res.writeHead(200, headers);
        res.end();
    }
});
                            
Now we must define how are we going to deal with the solution. We must identify each screen, obtain its dimensions, and when all the screens are connected we will have all the needed data (individual data and super-resolution).

When all the screens are connected we send the start signal with some data such as the super-resolution value and its child value (each screen resolution). "Super-resolution is the resolution created when we join all the screens in one"

How do we exchange data between the screens and the server? The answer is "Web Sockets". With web sockets, we can emit and listen to events. so when a screen connects to the server there is an event called "Connect", then the server sends an event to the screen sending the screen number (the server identifies the screen). The screen response is to give its width to the server. The server needs it to create the super-resolution. If all the screens are already identified then the server sends the start event to all the screens. That is where the data division takes place.

• Screen Side
We are using ThreeJS for the 3D visualization. ThreeJS is a cross-browser JavaScript library and application programming interface (API) used to create and display animated 3D computer graphics in a web browser using WebGL.

How can we divide the data between the screen?
The solution might seem to be long and complicated but it is not. ThreeJS has a setViewOffset method.

.setViewOffset ( fullWidth, fullHeight, x, y, width, height )
fullWidth — Super-resolution width
fullHeight — Super-resolution height
x — Horizontal offset of subcamera
y — Vertical offset of subcamera
width — width of the screen/canvas
height — height of the screen/canvas

• Start event (Screens)

The server tells the screen to set up their variables using the data they just have been provided (data explained above). They calculate its corresponding X-axis offset using the following code:


socket.on("start", (superRes) => {
    
    
    // super resolution width and height
    fullWidth = superRes.width;
    fullHeight = window.innerHeight;
    
    // calculate each screen startX
    let scRes = superRes.child;
    
    let keys = Object.keys(scRes);
    let arr = keys.map(Number).filter((e) => e % 2 != 0);
    
    startX = 0;
    
    if (screen % 2 != 0) {
        if (Math.max(...arr) == screen) {
            startX = 0;
        } else {
            for (let index = Math.max(...arr); index > screen; index -= 2) {
                startX += scRes[index];
            }
        }
    } else {
        for (let index = Math.max(...arr); index >= 1; index -= 2) {
            startX += scRes[index];
        }
        
        for (let index = 2; index < screen; index += 2) {
            startX += scRes[index];
        }
    }
    
    init();
    animate();
});
                            
• Data division function
                         
function View( canvas, fullWidth, fullHeight, viewX, viewY, viewWidth, viewHeight ) {
    canvas.width = viewWidth * window.devicePixelRatio;
    canvas.height = viewHeight * window.devicePixelRatio;
    
    const context = canvas.getContext("2d");
    
    // set perspective camera (you can also check the ortographic camera)
    camera = new THREE.PerspectiveCamera(20, viewWidth / viewHeight, 1, 10000);
    
    // set camera offset
    camera.setViewOffset(fullWidth, fullHeight, viewX, viewY, viewWidth, viewHeight);
    
    // camera default position
    camera.position.z = 1000; // default camera z index pos
    
    // override render function
    this.render = function () {
        // mouse event
        camera.position.x += ( mouseX - camera.position.x ) * 0.05;
        camera.position.y += ( - mouseY - camera.position.y ) * 0.05;
    
        renderer.setViewport(0, fullHeight - viewHeight, viewWidth, viewHeight);
        renderer.render(scene, camera);
    
        context.drawImage(renderer.domElement, 0, 0);
    };
}
                            
• Main setup
function init() {

    // Do not modify the following two lines of code
    const canvas1 = document.getElementById("canvas1");
    views.push(new View(canvas1, fullWidth, fullHeight, startX, 0, canvas1.clientWidth, canvas1.clientHeight));
    
    // ***********************************************************
    // ***********************************************************
    // YOUR CODE GOES HERE - Example in the template
    // ***********************************************************
    // ***********************************************************
    
    // do not modify the following 3 lines of code
    renderer = new THREE.WebGLRenderer({ antialias: true });
    renderer.setPixelRatio(window.devicePixelRatio);
    renderer.setSize(fullWidth, fullHeight);
    
}
                            

Future updates


We have accomplished our goals but there are always things that can be added.

  • Online Gamemode
  • PGN format chess play loader
The "Online Gamemode" can be easily done by using the firebase real-time database. You should create a game session that contains the following:
gamesessions: {
    session1: {
        UID_player1: ...,
        UID_player2: ...,
        Chessboard: ...,
        Turn: ...
    },
    ...
    sessionN: {
        UID_player1: ...,
        UID_player2: ...,
        Chessboard: ...,
        Turn: ...
    }
}
                            
This is an approximation, there are multiple ways to accomplish this

The "PGN format chess play loader" can be done by using the following utilities:
- PGN to JSON parser: Link
- Use the chess.js chess engine to reproduce the chess game.

Possible solution:
// "moves" is the array of moves obtained using the pgn to json parser.
// "chess" is the javascript chess engine
// EXAMPLE: chess.move('e4') returns -> { color: 'w', from: 'e2', to: 'e4', flags: 'b', piece: 'p', san: 'e4' }
// All you have to do now is:
//      - Create a parser from pgnJson to long algebraic notation
//      - Readapt the demo chess functions to take both notations.

// First option approximation
/**
 * @param {Array} moves : array of pgn format moves. ['e4', 'a6', ...]
 * @res {Array} res : array of long algebraic notation moves. ['E2E4', ...]
 */
const pgnToLongAlgebraic = (moves) => {
    let res = [];
    let chess = new Chess();
    moves.forEach(el => {
        const { from, to } = chess.move(el);
        res.push(from.toUpperCase() + to.toUpperCase())
    });

    return res;
}
                            

Conclusion

Google summer of code is a great opportunity to contribute to the open source community. Developing the first chess in the world that uses satellite communications is such a great way to improve the coding skills where I learned/improved:
• Build complex M2M environments
• Cloud computing
• SFTP Servers
• ReactJS skills

Contact info

Pablo Sanchidrian
PabloSanchi
pablo.sanchi.herrera@gmail.com