Overview

Cloud desktop is a cloud-based virtual desktop environment which you can access with just your browser. It allows you to run intensive robot simulations without your computer breaking down.

Features

• Ubuntu 18.04

• ROS Melodic

• Web-based GUI

Author: Julian Ho julianho (at) brandeis.edu

For new operators, please please please read all the documentations here before proceeding!

Questions and notes from Pito

• exactly how and where is "supervisor" used?

• Is the copy of tb3-ros in pitosalas still used for anything?

• Using lens.app to look at and work with Rover cluster

Overview

The cloud desktop architecture is simple. On a high level, it looks like this:

Cloud Desktop Cluster

Cloud desktop cluster is a cluster of cloud desktops. It is implemented as a K8s cluster for easy scheduling and orchestration of cloud desktop containers.

Cloud Desktop Container

The cloud desktop container provides a virtualized desktop environment that is isolated and portable. It consists of 3 components.

• VNC server paired with a NoVNC server

• VSCode server

• Tailscale client

For details, see .

Networking

K8s network:

• Used for communication with the load balancer to allow each container to be accessible from a URL

• Implemented with Flannel

Tailscale network:

• Used for communication between cloud desktops and robots globally

• Managed with

AWS Route53:

• Provides DNS records for redirecting traffic to the cluster

Cloud Desktop cluster is managed by Kubernetes. Specifically, it is a distro of Kubernetes named K3s.

Management

For cluster management, see Cluster Management.

Configurations

We use to run our k8s cluster. While leaving most configurations as default, we made the following customization,

• container runtime: docker

Namespaces

• clouddesktop-prod

  • Main namespace

  • Used for all active cloud desktops

Networking

The default networking backend is flannel with VXLAN as backend.

Ingress

The default ingress controller is traefik.

Storage

The default storage provider is local-path. In other words, we store all cloud desktop files locally on the node.

Node roles

As of May 2021

Overview

Each nodes in the cluster is setup in a similar way. The software stack on a node looks like this:

Container runtime

The default runtime is nvidia for all Nvidia-enabled nodes. For details, see /etc/docker/daemon.json.

Software versions

As of May 2021

Node hardwares

Desktop Creation

Desktop Destruction

All the source codes of Cloud Desktop is distributed in the following GitHub repos,

• Cloud Desktop Image https://github.com/pitosalas/tb3-ros

• Cloud Desktop K8s files https://github.com/campusrover/clouddesktop-k8s

• Standalone Cloud Desktop https://github.com/campusrover/clouddesktop-docker

Requirements

• kubectl Installation Guide

• terraform

• AWS Credentials to AWS Route53

  • Access Key

  • Secret Key

Setup

First obtain the users repo from .

Setup .env file by filling in all required fields:

It's better to use an IAM user group to create a new user associated with the clouddesktop user group. It will generate a access and secret key for you to put in the above file. The ingress IP is the ip address of the main node. Once everything is properly setup, do:

Setup terraform:

Common Operations

To under what each of these commands do under the hood, see .

Add a new user

id is the ID of the new user.

Delete a user

Warning: This will remove any persisted data!!

Change resources allocation for user

For user example, modify the file example-clouddesktop/deployment.yaml.

To increase minimum resource

For detailed explanation of what units you can change it to, see .

To increase maximum resource limit

For detailed explanation of what units you can change it to, see .

Add GPU support

Note: Beware if we have enough free GPUs in the cluster

Note: Make sure the docker image is a CUDA enabled variant (ie. tb3-ros:v2.1.1-cuda)

Apply changes

Warning: This will restart the cloud desktop container!!

To apply the previously changed values,

Restarting a Desktop

To restart a desktop, you need to delete and redeploy the desktop.

This will NOT lead to loss of data.

Access

Login to the provided link http://vnc.USER.ros.campusrover.org and enter your password (should be provided to you by TA).

Control

On the left you can find a panel for desktop control such as window scaling.

Left panel:

• Extra keys (ctrl-alt, esc key)

• Clipboard (use this for copy-and-paste)

• Fullscreen

On the bottom is the taskbar (similar to the one in Windows).

Window Scaling

To make sure the desktop fits your browser window, go to Settings -> Scaling Mode -> Choose Local Scaling.

Open Terminal

To open the terminal, click the start button at bottom-left corner, go to System Tools -> Choose LXTerminal.

Tailscale IP

Each clouddesktop has a special IP for connecting to our global private network. The tailscale IP can be found on the terminal.

Restart Desktop

To restart the clouddesktop in case of a freeze, open a terminal in Code Editor, type restart.

If that did not work, please reach out to your TA.

We administrate all our operations through the use of kubectl.

Prerequisites

Read up on the following materials,

Requirements

• kubectl

Setup

To access the K8s cluster, you will need to have a k3s.yaml credential file. It can be obtained by ssh into the master node of the cluster, under the directory /etc/rancher/k3s/k3s.yaml.

Once you have obtained the k3s.yaml file, make the following modification,

After the modification, this file is ready for use. Update your shell to always use this file,

To confirm it working,

Common Operations

See all nodes in cluster

See a specific node

See all deployed pods

• Notice that -n clouddesktop-prod refers to the clouddesktop-prod k8s namespace

See a specific pod

Draining a node

User

How do I add a new user?

Read User Management.

How do I delete a user?

Read .

How do I upgrade/downgrade a desktop?

Read .

How do I see all the active desktops in the cluster?

Image

How do I add a package to the image?

Read .

To make sure that new desktops are receiving this update, modify the deployment template under .

Node

Is it safe to restart a node?

Yes, but make sure to drain the node first. To drain the node,

After the reboot, k3s will be started automatically. If it is not started,

Troubleshooting

Desktop connectivity issue

If a group or all desktops are not connecting,

If only 1 desktop is not connecting,

Access

Login to the provided link http://code.USER.ros.campusrover.org and enter your password (should be provided to you by TA).

Cloud Desktop Container uses a custom docker image. The Dockerfile is located here.

Internals

Components

There are 3 main components in the container image,

• VNC server paired with a NoVNC server

• VSCode server

• Tailscale client

Defaults

Catkin Workspace: /my_ros_data/catkin_ws

Ports:

• novnc 80

• vnc 5900

• vscode 8080

Layers

The current container image is structured this way:

cosi119/tb3-ros

• Installs ROS melodic and ROS packages

• Installs custom packages used in class, like prrexamples

cosi119/ubuntu-desktop-lxde-vnc

• Provides a Ubuntu image with novnc and lxde preconfigured.

• Provides a CUDA enabled variant (image with -cuda tag suffix)

Process Management

Supervisord

Each of the components are managed by a process control system called supervisord. Supervisor is responsible for spawning and restarting these components. For detailed configs, see .

Modifing startup processes

Modify the supervisord.conf under tb3-ros/tb3-ros/files/supervisor/supervisord.conf.

Packages

Default packages

As of version 2.1.1,

• turtlebot3_msgs

• turtlebot3

• turtlebot3_simulations

Adding a new package

To add a package to the default catkin workspace, modify the Dockerfile under tb3-ros/tb3-ros/Dockerfile:

Github repo

Logging in for the first time

You should receive the following credentials for accessing your clouddesktop environment:

• Desktop link

  • e.g. http://vnc.julianho.ros.campusrover.org

• Code editor link

  • e.g. http://code.julianho.ros.campusrover.org

• Password

If you do not have those credentials, please reach out to your TA.

Accessing your cloud desktop

There are two ways to access your cloud desktop,

Data storage

Your data is securely saved on our servers and will persist on reboots.