I have been developing the C3D Web Vision component for more than three years. It is a modular client-server solution for 3D visualization in the browser. It can be easily integrated with any web app. I would like to share our experience and the tools we use for the C3D Web Vision development. Let us take a look at the graphics API of a browser, building the C++ web project, and microservice handling.
Getting Started
C3D Labs launched its web visualization project three years ago. Back then, we had extensive experience with desktop software development. Our goal was to transfer C3D Viewer to the web environment. We decomposed the solution into its key modules: the C3D Vision desktop visualization module, the C3D Modeler math module, C3D Converter, the business logic implemented in C++, and the GUI in Qt. C3D Modeler and C3D Converter components were converted into server applications. They run seamlessly on the backend without any modifications. the business logic had to be implemented in TypeScript to link it to the GUI. We used the Vue framework for the UI. We decided to keep our proprietary C3D Vision module as the visualization library but to build it as a web assembly. It was the biggest challenge so far.
TypeScript
The browser supports JavaScript, but most software developers don’t want to write code in JavaScript. It is a complicated language with dynamic memory allocation, not strictly typed, and with tons of hidden traps. Most developers use TypeScript. It is typed, indeed, and supports classes, and objects. And we are no exception, the entire frontend is written in TypeScript. Fig. 1, left shows a class implemented in TypeScript, and on the right, the result of building the TypeScript code in JavaScript.
No doubt, the TypeScript code looks nicer, while the JavaScript code would be a challenge to debug.
Both TypeScript and JavaScript support modules. Usually, any software project is divided into logical fragments: files, folders, and modules. You can do it in TypeScript, too. The only issue is that JavaScript can be run not only in the browser but also as a backend app using NodeJS. So there are so many kinds of JavaScript modules: commonjs, umd, amd... Fig. 2 shows some of the modules.
On the left there is a TypeScript code, and on the right there is the quite inconvenient umd module. Also, the module on the right required a dependency, a third-party library, to make it work in the browser.
Modules: commonjs, umd, amd
Developers can create and share new own modules (as we did with the visualization module). A module usually includes package.json, JavaScript itself, and a TypeScript definition. With a tool like Node Package Manager (npm), you can import external modules to your project and publish them to share with others. Fig. 3 shows a sample code that imports the modules using npm.
package.json is in the middle, below is devDependencies, where our C3D Vision wasm is imported. The code on the right shows how we use the imported module. So, we just import it as a regular module and we’re done. After the external modules are imported, the project folder contains the node_modules subfolder. All the dependent modules are uploaded to this subfolder during their installation.
We used Webpack to turn a large number of source files into a module. Webpack is a very powerful tool. It adds resources to projects, connects to the TypeScript compiler, reduces the script file size, adds different types of modules to the project, generates a browser or NodeJS project, etc.
Worker, Multithreading, and Promise
Sometimes developers do not realize the difference between multithreading and async codes. Async can be applied to single-threaded or multithreaded programming. A multithreaded app can also be synchronous or asynchronous. For example, when you watch a movie, the audio and video streams are processed in separate threads, but synchronously to make sure the audio is not going out of sync with the video. The browser executes any code in the same stream.
Fig. 4 shows Task 1:
Task 1 handles a download request (it downloads a file from the server), and after waiting for the download, Task 2 and Task 3 are launched. Using Promise, we can do it differently: start the downloading and the subsequent task concurrently. There are two threads in this case. Now let us take a look at another example.
If we have a time-consuming process, I recommend dividing it into smaller tasks (Fig. 5). Add periodic timeouts to enable the UI (e.g., to let the user interrupt the task). Otherwise, the browser may be frozen for twenty seconds and not respond to any requests, while the user would keep reloading the browser tab trying to understand why everything is freezing.
The browser can run multiple threads. Webworkers are provided for this purpose. Webworker is a separate thread or process (Fig. 6) to run long tasks separately from the main thread. There are two types of Webworkers: standard and shared. A shared Webworker can be accessed by multiple browser tabs. For instance, you’ve navigated to Yandex.Music in one tab, and then open another tab and control the music player there.
One more point about data sharing between threads. You cannot directly access data from another thread. Use messages to send (copy) the values. Large data structures (such as arrays) can be moved.
Local Store, Offline Mode
There is another type of Webworker called “service worker”. In contrast to the standard and shared workers, it runs even when the browser is offline. It supports offline data handling. For example, we connected to the LAN to open a 3D model, synchronized data with the server, and then picked up the laptop and went on a business trip. The service worker would save the data in the browser cache. When offline, it would work with a server providing the data from the browser cache. There are four data storage options:
- IndexDB is a database built into the browser
- LocalStorage stores key-value pairs
- SessionStorage also stores key-value pairs
- Storage can use up to 50% of the available disk space.
API for Web-Based Visualization
HTML5 has introduced the <canvas> tag to render 2D and 3D graphics. Now, WebGl and WebGL2 are available for 3D visualization. Since 2022, WebGL2 is officially supported on all devices, and it is recommended to switch to WebGL2. The WebGL API is nearly identical to OpenGL/ES, but some features are slightly different. For example, memory management is different, since the browser has some restrictions and such functions as glMapBufferRange are not available in web apps. The workaround is the glGetBufferSubData function, not available in OpenGL ES.
If you are familiar with OpenGL, you may know about the shared context. This feature can render the same model in two windows concurrently. Although the browser seems to also support the shared context (as specified in the docs), in reality, no browser supports it. Again, there is a workaround: the offscreen context. For example, we can draw not on the canvas, but on an image, and then render the image on the canvas. If you share the offscreen context using a shared web worker, you can render in two tabs simultaneously. It is very convenient for multiple monitor setups: you can display the controls in one tab, and visualize the 3D model in another tab separating GUI and visualization. (Fig. 7).
Suppose, we select the assembly components in the design tree on one tab and highlight the selected geometry in another tab.
WebGPU is scheduled for Q3 2023. It will offer more advanced graphic capabilities based on Vulcan and DirectX12. It is already available in the browser developer mode, but not for regular users.
C++ in the Browser
The most exciting part was to build a C++ project for the web. There is such a compiler as Emscripten. It supports C++ web apps. After compiling, we get JavaScript and WebAssembly files. WASM is an assembler for the browser. The browser runs it just like it runs JavaScript. Fig. 8 shows the C++ source code on the left and the initialization of the text output module.
The hello.js module is linked to the page as a global variable with the onRuntimeInitialized handler. The handler is called after WASM has been compiled by the browser and all the bindings have been initialized. Now we can use all the WASM functions.
Emscripten perfectly integrates with CMake and Conan. To import external Conan libraries for web builds, specify the WASM architecture in the recipe. To build dependencies for the web, add emsdk_installer to the dependency. During the build, Conan replaces the system compiler (e.g. gcc) with emcc. Emsdk includes header files that we can add to the project. For example, we can use OpenGL, and at the build stage, it replaces the OpenGL functions with the WebGL functions. And we still have a cross-platform code.
To call C++ functions from JavaScript, we have two binding options at the build stage: Embind (which we use), and WebIDl.
Fig. 9 lists the available types. Embind offers much more capabilities:
This is what the source code looks like (Fig. 10):
On the left, we defined a class for JavaScript using special macros. On the right is the definition in TypeScript. Since the build produces a JavaScript file, and we need TypeScript, we have to add a TypeScript definition manually.
Building a C++ Web Project: Possible Issues
Building a C++ project for the web creates a JavaScript module, that is, an export function that loads a WASM file to be compiled and returns Promise with the module. This is somewhat non-standard module behavior, so there are issues with its definition in TypeScript. To remove this obstacle, we implemented a wrapper that exports the object with the module initialization function. There are also issues with WASM file dependencies. As we provide a visualization library, the WASM file shall be always imported into the project. Webpack comes to the rescue. With it, the project will contain the WASM file as an ArrayBuffer.
There are a couple more details you should consider when building C++ projects for the web. First, call destructors of any objects created in your C++ module because the WASM memory is used as a project resource (heap emulation). JavaScript has a garbage collector, but it does not work for C++ objects, because the memory where the C++ objects are stored is a single array buffer. You can put up with it, just never forget to call the destructor.
Second, the memory size is limited to 2 Gb by default. It can be extended to 4 Gb by fine-tuning the compile options, but you can run into some OpenGl issues since there are errors in the compiler, specifically in the OpenGL module. We managed to patch the compiler fix these bugs, and get OpenGL working properly with 4 Gb. But the browser tabs are also limited to 4 Gb.
Third, calling C++ functions from JavaScript is slow, and it is mentioned in the docs. C++ is a typed language, and JavaScript is not, so there are tons of checks: whether a pointer or a variable is passed, etc. It slows down C++ calls from JavaScript. Suggestion: reduce C++ function calls from JavaScript to a minimum. We’ve also had issues with loading large amounts of data such as large assemblies. Such assemblies contain myriads of small objects, which is a productivity killer. We had to handle the entire in C++. All the implementation was already in C++. With TypeScript, we allocate some memory by calling external functions, put the loaded buffer there, and then serialize the C++ objects inside the WASM module. This has resulted in a significant performance gain.
There is also an issue with debugging tools. That is how the C++ code debugging looks in the browser (Fig. 11).
In the middle is a code fragment. We can add breakpoints and the run code to them, but there is no way to view the variable values. It is shown on the right (var 23, var 24). There is some call stack on the right, but it does not help much.
In this paper, I shared the experience with the development web graphics applications frontend using C3D Web Vision as an example. This is our approach to web development. I hope our story will be useful and help other web developers.
Author:
Sergei Klimkin
Team Lead of C3D Web Vision
C3D Labs
