diff --git a/librclone/README.md b/librclone/README.md index 66c68dce2..a95563209 100644 --- a/librclone/README.md +++ b/librclone/README.md @@ -4,13 +4,19 @@ This directory contains code to build rclone as a C library and the shims for accessing rclone from C and other languages. **Note** for the moment, the interfaces defined here are experimental -and may change in the future. Eventually they will stabilse and this +and may change in the future. Eventually they will stabilise and this notice will be removed. ## C The shims are a thin wrapper over the rclone RPC. +The implementation is based on cgo; to build it you need Go and a GCC compatible +C compiler (GCC or Clang). On Windows you can use the MinGW port of GCC, +installing it via the [MSYS2](https://www.msys2.org) distribution is recommended +(make sure you install GCC in the classic mingw64 subsystem, the ucrt64 version +is not compatible with cgo). + Build a shared library like this: go build --buildmode=c-shared -o librclone.so github.com/rclone/rclone/librclone @@ -20,9 +26,12 @@ Build a static library like this: go build --buildmode=c-archive -o librclone.a github.com/rclone/rclone/librclone Both the above commands will also generate `librclone.h` which should -be `#include`d in `C` programs wishing to use the library. +be `#include`d in `C` programs wishing to use the library (with some +[exceptions](#include-file)). -The library will depend on `libdl` and `libpthread`. +The library will depend on `libdl` and `libpthread` on Linux/macOS, unless +linking with a C standard library where their functionality is integrated, +which is the case for glibc version 2.34 and newer. ### Documentation @@ -33,9 +42,114 @@ For documentation see the Go documentation for: - [RcloneRPC](https://pkg.go.dev/github.com/rclone/rclone/librclone#RcloneRPC) - [RcloneFreeString](https://pkg.go.dev/github.com/rclone/rclone/librclone#RcloneFreeString) -### C Example +### Linux C example -There is an example program `ctest.c` with `Makefile` in the `ctest` subdirectory. +There is an example program `ctest.c`, with `Makefile`, in the `ctest` +subdirectory. It can be built on Linux/macOS, but not Windows without +changes - as described next. + +### Windows C/C++ guidelines + +The official [C example](#linux-c-example) is targeting Linux/macOS, and will +not work on Windows. It is very possible to use `librclone` from a C/C++ +application on Windows, but there are some pitfalls that you can avoid by +following these guidelines: +- Build `librclone` as shared library, and use run-time dynamic linking (see [linking](#linking)). +- Do not try to unload the library with `FreeLibrary` (see [unloading](#unloading)). +- Deallocate returned strings with API function `RcloneFreeString` (see [memory management](#memory-management)). +- Define struct `RcloneRPCResult`, instead of including `librclone.h` (see [include file](#include-file)). +- Use UTF-8 encoded strings (see [encoding](#encoding)). +- Properly escape JSON strings, beware of the native path separator (see [escaping](#escaping)). + +#### Linking + +Use of different compilers, compiler versions, build configuration, and +dependency on different C runtime libraries for a library and the application +that references it, may easily break compatibility. When building the librclone +library with MinGW GCC compiler (via go build command), if you link it into an +application built with Visual C++ for example, there will be more than enough +differences to cause problems. + +Linking with static library requires most compatibility, and is less likely to +work. Linking with shared library is therefore recommended. The library exposes +a plain C interface, and by using run-time dynamic linking (by using Windows API +functions `LoadLibrary` and `GetProcAddress`), you can make a boundary that +ensures compatibility (and in any case, you will not have an import library). +The only remaining concern is then memory allocations; you should make sure +memory is deallocated in the same library where it was allocated, as explained +[below](#memory-management). + +#### Unloading + +Do not try to unload the library with `FreeLibrary`, when using run-time dynamic +linking. The library includes Go-specific runtime components, with garbage +collection and other background threads, which do not handle unloading. Trying +to call `FreeLibrary` will crash the application. I.e. after you have loaded +`librclone.dll` into your application it must stay loaded until your application +exits. + +#### Memory management + +The output string returned from `RcloneRPC` is allocated within the `librclone` +library, and caller is responsible for freeing the memory. Due to C runtime +library differences, as mentioned [above](#linking), it is not recommended to do +this by calling `free` from the consuming application. You should instead use +the API function `RcloneFreeString`, which will call `free` from within the +`librclone` library, using the same runtime that allocated it in the first +place. + +#### Include file + +Do not include `librclone.h`. It contains some plain C, golang/cgo and GCC +specific type definitions that will not compile with all other compilers +without adjustments, where Visual C++ is one notable example. When using +run-time dynamic linking, you have no use of the extern declared functions +either. + +The interface of librclone is so simple, that all you need is to define the +small struct `RcloneRPCResult`, from [librclone.go](librclone.go): + +```C++ +struct RcloneRPCResult { + char* Output; + int Status; +}; +``` + +#### Encoding + +The API uses plain C strings (type `char*`, called "narrow" strings), and rclone +assumes content is UTF-8 encoded. On Linux systems this normally matches the +standard string representation, and no special considerations must be made. On +Windows it is more complex. + +On Windows, narrow strings are traditionally used with native non-Unicode +encoding, the so-called ANSI code page, while Unicode strings are instead +represented with the alternative `wchar_t*` type, called "wide" strings, and +encoded as UTF-16. This means, to correctly handle characters that are encoded +differently in UTF-8, you will need to perform conversion at some level: +Conversion between UTF-8 encoded narrow strings used by rclone, and either ANSI +encoded narrow strings or wide UTF-16 encoded strings used in runtime function, +Windows API, third party APIs, etc. + +#### Escaping + +The RPC method takes a string containing JSON. In addition to the normal +escaping of strings constants in your C/C++ source code, the JSON needs its +own escaping. This is not a Windows-specific issue, but there is the +additional challenge that native filesystem path separator is the same as +the escape character, and you may end up with strings like this: + +```C++ +const char* input = "{" +"\"fs\": \"C:\\\\Temp\"," +"\"remote\": \"sub/folder\"," +"\"opt\": \"{\\\"showHash\\\": true}\"" +"}"; +``` + +With C++11 you can use raw string literals to avoid the C++ escaping of string +constants, leaving escaping only necessary for the contained JSON. ## gomobile