Rclone for Dropbox on RaspberryPi (and other headless server)

Dropbox does not maintain their client for RaspberryPi, so you should find other ways to use it. There are some non-official clients, but I prefer using rclone, since it allows some interesting features for me:

  • Mount the remote as a local filesystem. This way I can expose a single Dropbox account to multiple users, for example, using the mounted Dropbox folder as an external storage in a Nextcloud server.
  • Sync files from and to Dropbox with different parameters, like limited bandwidth, parallel requests, etc.
  • It allows configuring the number of concurrent request to the remote, so you can speed up or reduce it accordingly to your requirements. In the case of Dropbox, a limit of 8-10 concurrent requests is enough to keep down the limit.

There is a guide on the rclone web site on how to configure a Dropbox remote, by creating a new Dropbox App. Follow the guidelines there but do not complete the authorization stage.

For a headless server the process is a little bit more tricky and requires you create a remote SSH tunnel to redirect a local port to your desktop running the ssh client. This way you will be able to run the OAuth2 workflow and authorize your remote headless server to access your Dropbox account.

In your desktop (or client) computer, run:

ssh -L localhost:53682:localhost:53682 your-user-name@your-remote-server.example.com

It will then open a session to your remote server that will tunnel requests on port 53682 to your desktop computer on port 53682.

Then run the following command on the remote-server:

rclone config reconnect remote-name:

Make sure you include the semicolon at the end, or rclone will complain. Here it is a sample of the output:

Already have a token - refresh?
y) Yes (default)
n) No
y/n> y

Use web browser to automatically authenticate rclone with remote?
 * Say Y if the machine running rclone has a web browser you can use
 * Say N if running rclone on a (remote) machine without web browser access
If not sure try Y. If Y failed, try N.

y) Yes (default)
n) No
y/n> y

2024/04/13 16:45:53 NOTICE: Make sure your Redirect URL is set to "http://localhost:53682/" in your custom config.
2024/04/13 16:45:53 NOTICE: If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth?state=nqmnwer1qn1m3n41m22n3413
2024/04/13 16:45:53 NOTICE: Log in and authorize rclone for access
2024/04/13 16:45:53 NOTICE: Waiting for code...

The command does not finish and it is waiting for you to complete the authorization flow on your desktop computer.

Now copy the URL that shows in the output and paste it in to a browser on your desktop computer: http://127.0.0.1:53682/auth?state=nqmnwer1qn1m3n41m22n3413

It will redirect you to a Dropbox Oauth screen. Log in, and then review the notice that shows about the authorization you are giving.

Once you complete the authorization, the access code is automatically fed into rclone configuration. If you have a look at the command on your remote server, you will see the rclone command has completed with a message like:

2024/04/13 16:45:53 NOTICE: Make sure your Redirect URL is set to "http://localhost:53682/" in your custom config.
2024/04/13 16:45:53 NOTICE: If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth?state=y01AK5m1jqC22vEm9DsMlw
2024/04/13 16:45:53 NOTICE: Log in and authorize rclone for access
2024/04/13 16:45:53 NOTICE: Waiting for code...
2024/04/13 16:46:13 NOTICE: Got code

Developing for Kstars

To build a stable or latest version of kstars, I recommend using the script from https://gitea.nouspiro.space/nou/astro-soft-build

The script creates two folder trees: one for the source code and another one for the build folders, that will include all the compiled objects and artifacts.

Once the script has created the build folder structure, you can manually build other parts of the application, like custom drivers or a new feature you would like to contribute.

I use the script to bootstrap the development and thus having a working copy of the app.

You can modify the original scripts and use your own repository or branch. Copy the original scripts and fine tune them for your repository and branch.

If you modify a driver, make sure to also edit file $INDIROOT/drivers.xml to sync the version.

Use make help to get a list of all available targets

You can also manually rebuild parts of the components. Here there are some tips.

Create build folder structure

You first need to create the building folder structure. This is the way the scripts create the build structure, so you can also make your own customization.

$ [ ! -d ../build-indi ] && \
 { cmake -B ../build-indi ../indi -DCMAKE_INSTALL_PREFIX=/usr -DCMAKE_BUILD_TYPE=Release || \
{ echo "INDI configuration failed"; exit 1; } }
$ cd ../build-indi

Workflow for testing a new version of a driver

In order to run your own version of a driver, you need to first build it. Here are the steps I follow:

  • Make your changes on the corresponding code.
  • Build the driver from the build folder.
  • Copy the driver to the driver install folder.

Use make help to get a list of all available targets in case of doubt.

For example, to build SnapCap driver:

$ cd ~/astro-soft-stable/build-indi/drivers/auxiliary

$ make indi_snapcap
[  0%] Built target hid
[  9%] Built target indicore
[ 22%] Built target indidevice
[ 22%] Built target eventloop
[ 31%] Built target dsp
[ 36%] Built target fpack
[ 95%] Built target indidriver_OBJECT
[ 95%] Built target indidriver
[100%] Built target indi_snapcap

To build a single driver:

  1. go to the build-* driver folder
  2. run make
  3. to install run sudo make install

For example, to build Rolloffino driver on the indi-3rdparties repository:

$ cd ~/astro-soft-stable/build-indi-3rdparty/indi-rolloffino 
$ make
Scanning dependencies of target indi_rolloffino
Building CXX object indi-rolloffino/CMakeFiles/indi_rolloffino.dir/rolloffino.cpp.o
Linking CXX executable indi_rolloffino
Built target indi_rolloffino

To install it you can use sudo make install or manually copy the driver to the driver folder (at /usr/bin for a RaspberryPi).

Start indiserver

It is better to manually start indiserver so you can see logs and other debugging information. For example, this will start a server listing on port 7624 and starting two drivers: Snapcap and Simultar filter wheel.

/usr/bin/indiserver -v -p 7624 -m 2048 -r 0 indi_snapcap  indi_simulator_wheel

Once you manually started the server, you can run kstars and create a profile to connect to the indiserver, in this case you should consider it a remote server running on your local computer. The profile should include the devices you used to manually start indiserver, and it is mandatory to include a CCD. In this example, the devices are Filter Simulator, SnapCap and the mandatory CCD, in this case, the Simulator.

Ejemplo de servidor TCP ESP8266

Hay montones de ejemplos de servidor TCP utilizando ESP8266WiFi. Están pensados para hacer peticiones sin estado, al estilo del protocolo HTTP. En este tipo de peticiones, el cliente abre la conexión, envía la petición, recibe la respuesta y luego cierra la conexión, por eso se dice que no tienen estado. Aqui hay un ejemplo de este código, para un servidor HTTP, que es un servidor sin estado (stateless).

El modelo de programación consiste en realizar la gestión de la transacción en el bloque loop del programa. En dicho bucle, se crea y se destruye el objeto cliente. Eso supone que el cliente tiene que volver a abrir la conexión en la siguiente conexión.

Este modelo no es adecuado para clientes que abren la conexión una vez y esperan que el servidor no la cierre. Un ejemplo de este tipo de conexiones es el de los drivers de indilib.

Si necesitas crear un servidor TCP que no cierre la conexión, es conveniente organizar el codigo de otra forma.

Aqui comparto una solución de ejemplo de código.

/*
   SocketServer.ino

    Created on: 4.11.2023
    Author: miceno.atreides@gmail.com

*/

#include <ESP8266WiFi.h>

#define MAX_BAUD_RATE 115200
String strData = "";             // a string to hold incoming data

uint16_t port = 8888;            // Port number
WiFiServer server(port);

// WiFi parameters
const char* ssid = "MOVISTAR_3887";
const char* password = "r5PP867mGjxKX329YhSj";
WiFiClient client;

void setup() {
  Serial.begin(MAX_BAUD_RATE);
  while (Serial.available() > 0)
    ;

  WiFi.disconnect();  //Prevent connecting to wifi based on previous configuration
  ESP.eraseConfig();

  WiFi.mode(WIFI_STA);  // WiFi mode station (connect to wifi router only)

  // Connect to WiFi
  WiFi.begin(ssid, password);
  Serial.println();
  while (WiFi.status() != WL_CONNECTED) {
    Serial.print(".");
    delay(200);
  }
  Serial.println("\nWiFi connected");
  Serial.print("START MIRRORING SERIAL: ");
  Serial.println(WiFi.localIP());

  // Start server
  server.begin();
}

void loop() {
  // Serial.println("looping..."); // DEBUG
  if (!client){
    // Only request a client in case there is no valid one.
    // The client should persist on the loop so it is a global variable.
    client = server.available();
  }

  if (client) {
    // Serial.println("client...");
    while (client.available() > 0) {
      char c;
      // Serial.println("data is available...");
      // read data from the connected client
      c = client.read();
      strData += c;
    }

    // Serial.println("\nNo more data available");
    if( strData.length()>0){
      Serial.printf("Message received is %s\n", strData.c_str());
      client.write("Response received");
      client.flush();
      strData = "";
    }
  }
}

La clave está en el código marcado:

if (!client){
    // Only request a client in case there is no valid one.
    // The client should persist on the loop so it is a global variable.
    client = server.available();
  }

En este caso, solamente debemos obtener un objeto WiFiClient si no tenemos uno disponible anteriormente. Se trata de obtener uno cuando sea necesario o cuando no esté disponible el anterior. Además, debemos asegurarnos de no llamar a client.stop() en el propio bucle, porque si no, cerraríamos la conexión y el cliente tendría que volver a abrirla.

Este patron de desarrollo se basa en la sobrecarga del operador bool en la clase WiFiClient.

Para verificar que esta solucion es robusta, puedes cargar el sketch anterior en un ESP8266, y por otro lado, desde un PC conectado a la misma WiFi, puedes construir un programa en Python para hacer de cliente. El programa Python abre la conexión con el servidor una única vez y envia continuamente peticiones.

De esta forma hemos construido un servidor TCP persistente, utilizando las librerías básicas de ESP8266. También puedes mejorar el codigo utilizando una librería asíncrona como ESPAsyncTCP.