Using mORMot with Azure Cosmos DB MongoDB

04 Mar 2023

mORMot is an "open source client-server ORM SOA MVC framework" for Delphi and FPC. Version 2 was recently released. Typical of such frameworks, mORMot supports many database drivers for connection to various well-known database products, one of which is MongoDB, for which mORMot provides a native wire protocol implementation in Pascal.

Microsoft Azure Cosmos DB is a brand of cloud-hosted database-as-a-service, a.k.a DBaaS. Cosmos DB supports a number of APIs / data models, or, in plain language, it provides hosted PostgreSQL, MongoDB, and several other products. For a few years now, Microsoft has been offering Try Azure Cosmos DB free, good for 30 days, with ability to extend or convert to paid account.

In this and subsequent posts, we'll look at using mORMot with Azure Cosmos DB.

First, sign up with Azure Cosmos DB, then choose to create an Azure Cosmos DB for MongoDB. When done, follow the link to the Azure portal to manage the database instance. Take note of the Settings menu on the side bar on the left.

Azure Cosmos DB Settings

Click on Connection strings to get the information required to connect to our MongoDB instance. Take note of HOST, PORT, USERNAME and PRIMARY PASSWORD fields in the default Read-write Keys tab. Note that SSL (more accurately, TLS) is mandatory, for obvious reasons.

Azure Cosmos DB Connection String

Next, we will modify mORMot's bundled MongoDB example minimally to use our Azure MongoDB instance. In MongoDBTests.dpr, add mormot.lib.openssl11 to the uses clause. All other changes will be made to MongoDBTestCases.pas; below, all line numbers refer to the original source file:

  • Line 11, define TESTMONGOAUTH.

  • Lines 39 and 41, change MONGOSERVER and MONGOPORT to the connection string's HOST and PORT values.

  • Line 181, change DB_NAME to 'mormot'.

  • Lines 182 and 183, change USER_NAME and USER_PWD.

  • Comment out lines 208-218. Instead of creating a new MongoDB user, we'll use the pre-created user for this demo.

  • Change line 220 and add following lines to enable TLS:

  fClient := TMongoClient.Create(MONGOSERVER, MONGOPORT, [mcoTls]); // This is line 220
  with fClient.ConnectionTlsContext do
    begin
      Enabled := true;
      IgnoreCertificateErrors := false;
    end;
  • Change line 233 (remember, line number refers to original file) to following:
  fDB := fClient.OpenAuth(DB_NAME, USER_NAME, USER_PWD, true); 
  • Change line 462 and add following lines to enable TLS:
  fMongoClient := TMongoClient.Create(MONGOSERVER, MONGOPORT, [mcoTls]); // This is line 462
  with fMongoClient.ConnectionTlsContext do
    begin
      Enabled := true;
      IgnoreCertificateErrors := false;
    end;
  • Change line 472 (again, this is line number in original file) to following:
  fDB := fMongoClient.OpenAuth(DB_NAME, USER_NAME, USER_PWD, true); 

And that's it. I have posted the complete set of files required to build this demo to my mormot-cloud repo.

Rebuild and run:

% lazbuild MongoDBTests.lpi
...
(1008) 2758 lines compiled, 1.1 sec, 3989044 bytes code, 2500148 bytes data
(1022) 3 hint(s) issued
(1023) 1 note(s) issued

% ./MongoDBTests

   Mongo DB
  ----------


1. Direct access

 1.1. Direct: 
  - Connect to local server: 6 assertions passed  3.25s
  - Drop and prepare collection: 6 assertions passed  239.51ms
  - Fill collection: 353 assertions passed  33.41s
     100 rows inserted in 32.40s i.e. 3/s, aver. 324.03ms, 0.9 KB/s
  - Drop collection: no assertion  510.53ms
  - Fill collection bulk: 2 assertions passed  2.56s
     100 rows inserted in 2.25s i.e. 44/s, aver. 22.51ms, 4.3 KB/s
  - Graceful reconnect: 3 assertions passed  2.16s
  - Read collection: 603 assertions passed  3m07
     100 rows read at once in 892.18ms i.e. 112/s, aver. 8.92ms, 9.1 KB/s
  - Update collection: 136 assertions passed  1m01
     100 rows updated in 30.67s i.e. 3/s, aver. 306.79ms, 793 B/s
  - Delete some items: 82 assertions passed  31.43s
     20 rows deleted in 6.37s i.e. 3/s, aver. 318.57ms, 514 B/s
  Total failed: 0 / 1,191  - Direct PASSED  5m22


2. ORM

 2.1. ORM: 
  - Connect to local server: 6 assertions passed  3.13s
  - Insert: 102 assertions passed  33.87s
     100 rows inserted in 32.95s i.e. 3/s, aver. 329.58ms, 755 B/s
  - Insert in batch mode: 104 assertions passed  2.14s
     100 rows inserted in 1.84s i.e. 54/s, aver. 18.47ms, 7 KB/s
  - Retrieve: 1,001 assertions passed  31.26s
     100 rows retrieved in 31.26s i.e. 3/s, aver. 312.62ms, 1.3 KB/s
  - Retrieve all: 901 assertions passed  325.37ms
     100 rows retrieved in 325.32ms i.e. 307/s, aver. 3.25ms, 39.8 KB/s
  - Retrieve one with where clause: 1,000 assertions passed  31.63s
     100 rows retrieved in 31.63s i.e. 3/s, aver. 316.37ms, 1.3 KB/s
  - Retrieve from SQL: 4,709 assertions passed  8.30s
     535 rows retrieved in 8.30s i.e. 64/s, aver. 15.52ms, 8.2 KB/s
  - Update: 902 assertions passed  31.29s
     100 rows updated in 30.98s i.e. 3/s, aver. 309.81ms, 0.9 KB/s
  - Blobs: 2,707 assertions passed  2m03
     100 rows updated in 30.99s i.e. 3/s, aver. 309.96ms, 638 B/s
  - Delete: 843 assertions passed  8.89s
     20 rows deleted in 5.52s i.e. 3/s, aver. 276.01ms, 480 B/s
  - Delete in batch mode: 743 assertions passed  544.53ms
     20 rows deleted in 307.30ms i.e. 65/s, aver. 15.36ms, 1 KB/s
  Total failed: 0 / 13,018  - ORM PASSED  4m35

Using MongoDB 3.6.0 
Running on Ubuntu 20.04.5 LTS - Linux 5.4.0-144-generic
Compiled with mORMot 2.0.4952
Generated with: Free Pascal 3.3.1 64 bit Linux compiler

Time elapsed for all tests: 9m57
Performed 2023-03-04 17:24:04 by pierce

Total assertions failed for all test suits:  0 / 14,209
! All tests passed successfully.

Memory Usage Report:
 Flags: SERVER  assumulthrd erms debug repmemleak
 Small:  2K/200KB  including tiny<=128B arenas=8 fed from Medium
 Medium: 3MB/3MB    peak=3MB current=3 alloc=3 free=0 sleep=0
 Large:  0B/0B    peak=0B current=0 alloc=0 free=0 sleep=0
 Small Blocks since beginning: 108K/9MB (as small=43/46 tiny=55/56)
  48=30K  32=27K  64=10K  128=8K  112=7K  192=4K  80=4K  240=2K
  96=2K  176=2K  144=2K  160=1K  224=896  208=700  528=669  256=468
 Small Blocks current: 2K/200KB
  32=1K  48=689  352=176  64=81  112=70  128=68  80=41  96=38
  880=7  416=7  160=6  192=6  624=4  224=4  1264=3  576=3

%

Took about 10 minutes to run. Not very fast... To be fair, default region for Try Cosmos DB for free is North America, and I am in Southeast Asia.

Go back to Azure portal, click on Data Explorer, and marvel at the data just inserted:

Azure Cosmos DB MongoDB Data Explorer

In a subsequent post we shall see how to avoid hardcoding our database password in the source code.

Tags: Azure, MongoDB, mORMot, Pascal

dwindows with Free Pascal for iOS and Android

30 Dec 2022

Dynamic Windows, aka dwindows, is a cross-platform GUI application toolkit with a C API. dwindows works with GTK2, GTK3, Windows API, OS/2 Presentation Manager, Cocoa (macOS and iOS), and Android.

Having a C API makes it possible to build bindings for other programming languages, like Pascal. Free Pascal already powers Lazarus, a rapid application development (RAD) tool for building cross-platform GUI applications in Pascal. There is a large overlap between the platforms covered by dwindows and Lazarus, with the notable exception of iOS.

Free Pascal is able to produce binaries for many processor architectures, including for the ARM CPU used in Android and iOS devices. For Lazarus, there is the excellent LAMW - the Lazarus Android Module Wizard - which extends Lazarus for building Android apps with native look-and-feel.

What Free Pascal and Lazarus don't have currently is a good iOS story. And here the C API of dwindows offers possibilities. Perhaps, it becomes feasible to build Android and iOS apps using a common Pascal code base for most of the required functionality.

At the moment, I've put together the littlest Pascal demo mobile app for dwindows. Here's a screenshot, Android phone on the left, iPhone on the right.

Pascal Hello World on Android and iOS with dwindows

The code putting up the message box is the same:

procedure dwmain(argc: Integer; argv: PPChar); cdecl;
begin
  dw_init(TRUE, argc, argv);
  dw_messagebox('Free Pascal + dwindows', DW_MB_OK or DW_MB_INFORMATION, 'Hello from Pascal, courtesy of dwindows!');
  dw_exit(0);
end;

Scaffolding is different: on Android, the Pascal program is built as a shared library, dwindows is another shared library, while on iOS the Pascal program becomes an executable and dwindows is a framework.

My next step is to build a demo with more GUI widgets.

Beyond the short term, as I see it, the best approach to building Android/iOS apps with a common Pascal code base is to use HTML/CSS through the web widget, like what my fpwebview does for the desktop platforms. (Indeed, a recurring question for webview is whether it works for Android and iOS.)

Using, say, mORmot to do 'server-side rendering' fully, the app requires minimal to zero Javascript on the front end. Concerns about latency, state management, etc. that motivated the creation of fancy Javascript front-end frameworks go away as the app is effectively a single-user single-browser tab on-device web app.

Of course, the modern mobile phone is a powerful device, and a mobile app's requirements could go far beyond simply showing stuff on screen and accepting input via touch and virtual keyboard. To build fully fledged iOS apps with Free Pascal, there is probably no shortcut to building Pascal interfaces to the platform APIs for cameras, audio I/O, accelerometer, GPS, etc.

Tags: Android, dwindows, iOS, Pascal

fpwebview embedded in Lazarus GUI

22 Nov 2022

Lazarus is an IDE-cum-widget set interface for developing cross platform GUI applications. I've now implemented a demo of using fpwebview with Lazarus, i.e., embedding webview in a desktop Lazarus GUI application.

Here's a macOS screenshot:

fpwebview embedded in Lazarus Cocoa app demo

In the screenshot, the three buttons above the web content are outside of the webview display area, and are Lazarus components making up the GUI application. The 'Say Yo To Webview' button demonstrates updating content shown by webview in response to GUI action, while the 'Lazarus Say Hello' button demonstrates standard GUI functionality.

Technically, this application demonstrates concurrent execution of the Lazarus GUI event loop managing the application's windows, and webview's own event loop that manages the Lazarus window in which webview is embedded.

The demo works on macOS (using Lazarus Cocoa widget set) and Windows (using Lazarus Windows widget set).

For Linux:

  • with Lazarus GTK2: Impossible, as GTK2 and GTK3 (which webview on Linux uses) cannot coexist in one application (that executes as one operating system process).

  • with Lazarus GTK3: Should be trivial. Note that at present Lazarus GTK3 is work-in-progress and not mature.

  • with Lazarus Qt5/Qt6: Possible, potentially tricky. The basic requirement is to convert a Qt window handle into a GTK3 window suitable for use with webview.

Tags: fpwebview, Pascal

Pascal WebAssembly + pas2js

09 Apr 2022

WebAssembly (abbreviated Wasm) is "a binary instruction format for a stack-based virtual machine. Wasm is designed as a portable compilation target for programming languages, enabling deployment on the web for client and server applications." Currently, all major web browsers support Wasm. Standalone and embeddable Wasm runtimes also exist that allow running Wasm programs outside of the web browser.

So far, Wasm seems be have been of more interest among the AOT compile-to-native programming languages. Free Pascal, which works across a large number of processor architectures, operating systems, and embedded platforms, also support Wasm compilation targets; see this wiki page on setting it up. Additionally, Free Pascal also comes with a Pascal-to-Javascript transpiler known as pas2js.

I've put up a reworked Free Pascal Wasm demo of Conway's Game of Life. The game play was implemented by Dmitry Boyarintsev, in Pascal, and compiled to Wasm. The user interface is implemented by me, in Pascal, and transpiled to Javascript. For good measure, I've included a simple web server, also implemented in Pascal, that serves the Wasm, Javascript and other static content.

So, to build all Pascal programs in the demo from source, three Free Pascal compilation targets are used:

  • the native compiler, for the web server,
  • pas2js, for the user interface,
  • the Wasm compiler, for the game play.

Here's a screenshot of the user interface running on my desktop web browser in mobile-responsive mode:

pas2js Conway Game of Life

Tags: Pascal, WebAssembly

fpwebview Embedded Web Server

17 Feb 2022

I've implemented an embedded localhost web server demo for fpwebview. The demo runs on Linux macOS, and Windows 10. Here's a Linux screenshot:

fpwebview Embedded Server Demo

The program consists of a multi-threaded web server listening on localhost:8000, and the web browser widget showing the content, styled by CSS, all loaded from said web server.

Because the embedded web server is listening on localhost:8000, you could visit it with a regular web browser running on the same machine. However, in the demo program, the buttons work by using webview APIs to perform what is effectively in-process RPC between Javascript and Pascal. From a regular web browser, clicking those buttons will have no effect, since the browser's Javascript runtime has nothing to do with the Pascal procedures implementing the buttons' functionalities.

For serious usage beyond a demo, additional security measures are possible to ensure that the embedded web server serves only the web browser widget and no other client.

Tags: fpwebview, Pascal

fpwebview Bidirectional Javascript-Pascal Calling

16 Feb 2022

I've implemented bidirectional Javascript-Pascal calling for fpwebview. Here's the demo program:

program js_bidir;

{$linklib libwebview}

{$mode objfpc}{$H+}

uses
  {$ifdef unix}cthreads,{$endif}
  math,
  webview;

var
  w: PWebView;
  html: String;

procedure SayHello(const seq: PAnsiChar; const req: PAnsiChar; arg: Pointer); cdecl;
var
  s: String;
begin
  s := 'var p = document.createElement("p")' + LineEnding +
    'p.innerHTML = "Yo again!"' + LineEnding +
    'document.body.appendChild(p)';
  webview_eval(w, PAnsiChar(s));
  webview_return(w, seq, WebView_Return_Ok, '{result: "<p>Yo!</p>"}');
end;

begin
  { Set math masks. libwebview throws at least one of these from somewhere deep inside. }
  SetExceptionMask([exInvalidOp, exDenormalized, exZeroDivide, exOverflow, exUnderflow, exPrecision]);

  html := 'data:text/html,<html>' + LineEnding +
    '<head></head>' + LineEnding +
    '<body><button onClick="SayHello()">Say Hello</button>' + LineEnding +
    '<div id="greeting"></div>' + LineEnding +
    '<script>var SayHello = function() {' + LineEnding +
    '  HostSayHello().then((x) => document.getElementById("greeting").innerHTML = x.result)' + LineEnding +
    '}</script>' + LineEnding +
    '</body>' + LineEnding +
    '</html>';

  w := webview_create(WebView_DevTools, nil);
  webview_set_size(w, 700, 200, WebView_Hint_None);
  webview_set_title(w, PAnsiChar('WebView - Pascal Javascript Bidirectional'));
  webview_bind(w, PAnsiChar('HostSayHello'), @SayHello, nil);
  webview_navigate(w, PAnsiChar(html));
  webview_run(w);
  webview_destroy(w);
end.

In the program, the Pascal procedure SayHello is made accessible to Javascript as HostSayHello. When Javascript calls HostSayHello, the Pascal code in turns invokes Javascript to update the DOM, then returns a response to the Javascript caller.

Next up, an embedded web server to serve HTML, CSS and other content including Javascript source.

Tags: fpwebview, Pascal

fpwebview - webview in Pascal

14 Feb 2022

Webview is a "tiny cross-platform webview library for C/C++/Golang to build modern cross-platform GUIs." Webview uses Cocoa/WebKit on macOS, gtk-webkit2 on Linux, and Edge on Windows 10.

There exist various language bindings for Webview, as listed on Webview's Github page. I've just published my Free Pascal binding here.

Here's a simple Pascal program that uses Webview to implement a web browser.

program browser_cli;

{$linklib libwebview}

uses
  {$ifdef unix}cthreads,{$endif}
  math,
  webview;

var
  w: PWebView;

begin
  { Set math masks. libwebview throws at least one of these from somewhere deep inside. }
  SetExceptionMask([exInvalidOp, exDenormalized, exZeroDivide, exOverflow, exUnderflow, exPrecision]);

  writeln('Hello, webview, from Pascal!');
  w := webview_create(WebView_DevTools, nil);
  webview_set_size(w, 1024, 768, WebView_Hint_None);
  webview_set_title(w, PAnsiChar('WebView Free Pascal'));
  webview_navigate(w, PAnsiChar('https://www.freepascal.org/'));
  webview_run(w);
  webview_destroy(w);
  writeln('Goodbye, webview.');
end.

Coincidentally, Toma┼ż Turk recently announced Pharo-WebView.

Tags: fpwebview, Pascal

Embedding Pharo in Pascal HOWTO - Linux

23 Jan 2020

Pharo headless VM

First, build the Pharo headless VM, following the instructions in its README. When done, the VM files are in build/vm.

Pharo application image

Next, prepare the Pharo application image. Basically, start from a fresh Pharo 8.0 image and load the EmbeddedSupport code from Pablo's repository. The following snippet works, assuming you have cloned Pablo's repository locally; adjust the gitlocal path in the snippet, of course.

Metacello new
	baseline: 'EmbeddedSupport';
	repository: 'gitlocal:///home/pierce/src/st/pharo-vm-embedded-example/smalltalk-src';
	load.

NoChangesLog install.
NoPharoFilesOpener install.
PharoCommandLineHandler forcePreferencesOmission: true.

SmalltalkImage current snapshot: true andQuit: true

Windows resource

Then, embed the Pharo application image into a Windows resource file.

Here's a simplified version of resources.rc. This assumes you have named the image to be embedded Pharo.image.

300 RCDATA "Pharo.image"

Building a Windows resource file on Linux requires windres. On Ubuntu, this program is part of the package binutils-mingw-w64-x86-64.

Place the application Pharo.image and resources.rc in the same directory. Then,

% /usr/bin/x86_64-w64-mingw32-windres -i resources.rc -o resources.res
% ls -l
total 113488
-rw-r--r-- 2 pierce pierce 58098672 Jan 23 14:10 Pharo.image
-rw-r--r-- 2 pierce pierce       25 Jan 23 14:01 resources.rc
-rw-r--r-- 2 pierce pierce 58098736 Jan 23 14:10 resources.res

The output resources.res is the compiled resources file.

Pascal host program

Finally we are ready to build the Pascal host program. Move resources.res to this repository's embedded-cli directory. Also create a link to libPharoVMCore.so or make a copy of it. The directory's content is now as follows:

% ls -l
total 58736
-rwxr-xr-x 2 pierce pierce  3344760 Jan 23 09:26 libPharoVMCore.so*
-rw-r--r-- 2 pierce pierce 58098736 Jan 23 14:10 resources.res
-rw-r--r-- 4 pierce pierce     1945 Jan 23 12:55 runPharoEmbedded.pas
-rw-r--r-- 4 pierce pierce     3824 Jan 23 12:54 ulibEmbeddedImage.pas
-rw-r--r-- 5 pierce pierce     1045 Jan 22 15:42 ulibPharoVM.pas

At the start of runPharoEmbedded.pas, the line {$R resources.res} is the Pascal compiler directive to incorporate resources.res into the executable that is being built:

program runPharoEmbedded;

{$mode objfpc}{$H+}
{$R resources.res} { <= compiler directive to incorporate resources.res }
{$linklib m}

Build the host program - the Pascal compiler is its own make system and knows to compile the necessary source files. (The Pascal compiler also knows how to invoke windres to compile resources.rc into resources.res when so configured. I've done that part by hand since this is a HOWTO.)

% fpc runPharoEmbedded.pas
Free Pascal Compiler version 3.0.4 [2018/10/29] for x86_64
Copyright (c) 1993-2017 by Florian Klaempfl and others
Target OS: Linux for x86-64
Compiling runPharoEmbedded.pas
Compiling ulibPharoVM.pas
Compiling ulibEmbeddedImage.pas
Compiling resource runPharoEmbedded.or
Linking runPharoEmbedded
/usr/bin/ld: warning: link.res contains output sections; did you forget -T?
232 lines compiled, 0.4 sec

% ls -l runPharoEmbedded
-rwxr-xr-x 1 pierce pierce 58884328 Jan 23 14:22 runPharoEmbedded*

Much of the size of the executable is due to the embedded Pharo.image.

We'll run runPharoEmbedded in the headless VM build directory because Pharo's baked-in library lookup currently requires this. (More on this in a later post.) So move the program over.

% mv runPharoEmbedded ~/src/st/opensmalltalk-vm-pharo/build/vm
% cd ~/src/st/opensmalltalk-vm-pharo/build/vm
% ls -T 30
libB2DPlugin.so*             libgit2.so.0.25.1             libSDL2-2.0.so.0*      libssl.so*
libBitBltPlugin.so*          libgit2.so.25                 libSDL2-2.0.so.0.7.0*  libssl.so.1.0.0*
libcrypto.so.1.0.0*          libIA32ABI.so*                libSDL2.so*            libSurfacePlugin.so*
libDSAPrims.so*              libJPEGReaderPlugin.so*       libSecurityPlugin.so*  libTestLibrary.so*
libffi.so*                   libJPEGReadWriter2Plugin.so*  libSocketPlugin.so*    libUnixOSProcessPlugin.so*
libffi.so.7*                 libLargeIntegers.so*          libSqueakFFIPrims.so*  libUUIDPlugin.so*
libffi.so.7.1.0*             libLocalePlugin.so*           libSqueakSSL.so*       pharo*
libFileAttributesPlugin.so*  libMiscPrimitivePlugin.so*    libssh2.so*            runPharoEmbedded*
libFilePlugin.so*            libPharoVMCore.so*            libssh2.so.1*
libgit2.so                   libPThreadedPlugin.so*        libssh2.so.1.0.1*

Set up LD_LIBRARY_PATH. The first path segment is for the Pharo VM. The second is for libcairo2.so needed by the embedded Pharo.image - on Ubuntu, it lives in /usr/lib/x86_64-linux-gnu, which isn't in Pharo 8's current hardcoded lookup path. Then run the executable:

% export LD_LIBRARY_PATH=`pwd`:/usr/lib/x86_64-linux-gnu
% uname -a
Linux Otrus 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
% ./runPharoEmbedded
lala

runPharoEmbedded

Ta da! "lala" is printed by Pharo.

Tags: embedded, FFI, headless, Pascal

Cross Platform Embedded Pharo

22 Jan 2020

Recently Pablo Tesone published an example of embedding a Pharo image as a Windows resource inside a program written in C.

Well, the excellent and, like Smalltalk, vastly underrated Free Pascal programming system has cross-platform support for handling Windows resources. With it, I have reimplemented Pablo's example: One source code building executables that run on Ubuntu 18.04 and MacOS Mojave. Not tested on Windows yet, but I'm sure it'll work there too. The three Pascal source files, including blank lines and comments, total 209 lines:

  • ulibPharoVM.pas, FFI to libPharoVMCore, comprising one data type and two functions.

  • ulibEmbeddedImage.pas, implementing the functions called by libPharoVMCore to access the Pharo image that is embedded as a Windows resource.

  • runPharoEmbedded.pas, the main program which is pretty much a straight port of Pablo's C program.

Repository on GH. More details to come.

Tags: embedded, FFI, headless, Pascal

Metaprogramming Pascal with Mustache in Smalltalk

06 Feb 2016

A while back I wrote about command line scripting of Pharo to backup an SQLite database, specifically, this server's iptables log. Eventually I wrote a program in Pascal to archive and rollover the log database. Reason being, using the excellent Free Pascal compiler, I link SQLite statically into the final executable, meaning I only deploy a single binary executable, which is a tad more convenient than deploying the Pharo VM, image, sources, and script files.

Well, not quite just one file.

My program, iptlb, uses the SQLite online backup API to back up the running log database to an archive database file, then "backs up" an empty template database to the running log database file, effectively overwriting it. Because I want the flexibility to change the database schema during each backup run, iptlb creates a new template database each time it is invoked. Because I also want just one Pascal source file, I want to store the database schema in the source file itself, so that the schema gets version-controlled along with the Pascal code.

However, Pascal does not support multi-line string literals:

(* This Pascal code is invalid. *)
const
  dbSchema = '
    create table x (xk int, xv varchar);
    create table y (yk int, yv int);
  ';

This means that I cannot embed the database schema into iptlb's Pascal source directly as a multi-line string. In production, I have to also deploy the database schema SQL as a separate file which is read by iptlb to create its template database.

There is a workaround in Pascal for lack of multi-line strings, as follows:

var
  sl: TStringList;
  dbSchema: String;
begin
  sl := TStringList.create;
  sl.add('create table x (xk int, xv varchar);');
  sl.add('create table y (yk int, yv int);');
  dbSchema := sl.text; // dbSchema is now the multiline string.
  sl.free;
end;

This looks like a templating thing. And that calls for Mustache!

After fiddling around, I created a simple class PMMultiLineStringTemplate which wraps the work with Mustache. Here's the motivating use case:

| tmpls mst |

"The Mustache template of the Pascal code."
tmpls := '
function {{functionName}}: String;
var
  sl: TStringList;
begin
  sl := TStringList.create;{{#stringList}}
  sl.add(''{{sline}}'');{{/stringList}}
  {{functionName}} := sl.text;
  sl.free;
end;
'.

mst := PMMultilineStringTemplate new.
mst atAttribute: 'functionName' putValue: 'dbSchema'.
mst template: tmpls.

FileSystem disk workingDirectory / 'ulog-schema.sql' readStreamDo: [ :rs |
    rs ascii.
    [ rs atEnd ] whileFalse: [
        | x |
        x := rs nextLine.
        (x size > 0) ifTrue: [
            mst atAttribute: 'stringList' withSubKey: 'sline' putValue: x ]]].

FileSystem disk workingDirectory / 'dbschema.pas' writeStreamDo: [ :ws |
    ws ascii;
        truncate;
        lineEndConvention: #lf; 
        nextPutAll: mst value ]

Copying the output of dbschema.pas (generated with a proper schema as above) into iptlb.pas, I now have one source file iptlb.pas embedding the SQL schema, and I deploy one binary executable iptlb only.

Tags: metaprogramming, Pascal