Monday, September 26, 2016

WebApp for converting local date time to GPS week and seconds of week

This online calculator can convert the local date time values into the equivalent GPS week and seconds of week. The GPS leap seconds from 1980 till 2016 are taken into account in the conversion.


To use this calculator,
  1. In the Settings pane on the right, select your local time zone, e.g. GMT+0800.
  2. Choose the local date time format, e.g. YYYY-MM-DD HH:MM:SS.
  3. Choose the output delimiter format, e.g. comma.
  4. In the Input local time field, type in one or more local date time values.
  5. Click the Convert to GPS Week Seconds button.

    The converted results are shown in the output field below.

Monday, September 19, 2016

Create a Windows Installer using CMake and CPack

CMake can be used to create a Windows installer with CPack and NSIS (Nullsoft Scriptable Install System). NSIS can be downloaded from http://nsis.sourceforge.net. This post follows and extends the tutorial from the CMake Wiki at https://cmake.org/Wiki/CMake:Component_Install_With_CPack.

I wanted to build a Windows installer for a 64 bit Visual Studio application that was build-managed using CMake. This post makes use of the example source code downloadable from https://cmake.org/Wiki/File:ComponentExampleStart.zip.

Add in CPACK macros to CMakeLists.txt
  1. Download and extract the example zip file into a folder e.g. D:\Temp\ComponentExampleStart\ as shown below. I then created a build sub-folder build underneath the D:\Temp\ComponentExampleStart\Source\ folder.
  2. Use a text editor and edit the CMakeLists.txt file to add in CPACK macros according to the Wiki. An edited example is shown below.
cmake_minimum_required(VERSION 2.6.0 FATAL_ERROR)
project(MyLib)

add_library(mylib mylib.cpp)

add_executable(mylibapp mylibapp.cpp)
target_link_libraries(mylibapp mylib)

install(
  TARGETS mylib 
  ARCHIVE
  DESTINATION lib
  COMPONENT libraries
  )
install(
  TARGETS mylibapp
  RUNTIME
  DESTINATION bin
  COMPONENT applications
  )
install(
  FILES mylib.h
  DESTINATION include
  COMPONENT headers
  )
#
# CPACK macros below here
#
set (CPACK_PACKAGE_NAME "MyLib")
set (CPACK_PACKAGE_VENDOR "CMake.org")
set (CPACK_PACKAGE_DESCRIPTION_SUMMARY "MyLib - CPack Component Installation Example")
set (CPACK_PACKAGE_VERSION "1.0.0")
set (CPACK_PACKAGE_VERSION_MAJOR "1")
set (CPACK_PACKAGE_VERSION_MINOR "0")
set (CPACK_PACKAGE_VERSION_PATCH "0")
set (CPACK_PACKAGE_INSTALL_DIRECTORY "CPack Component Example")

# Define components and their display names
set (CPACK_COMPONENTS_ALL applications libraries headers)
set (CPACK_COMPONENT_APPLICATIONS_DISPLAY_NAME "MyLib Applications")
set (CPACK_COMPONENT_LIBRARIES_DISPLAY_NAME "Libraries")
set (CPACK_COMPONENT_HEADERS_DISPLAY_NAME "C++ Headers")

# Human readable component descriptions
set (CPACK_COMPONENT_APPLICATIONS_DESCRIPTION
  "An extremely useful application that makes use of MyLib")
set (CPACK_COMPONENT_LIBRARIES_DESCRIPTION
  "Static libraries used to build programs with MyLib")
set (CPACK_COMPONENT_HEADERS_DESCRIPTION
  "C/C++ header files for use with MyLib")

# Define dependencies between components
set (CPACK_COMPONENT_HEADERS_DEPENDS libraries)

# Define groups
set(CPACK_COMPONENT_APPLICATIONS_GROUP "Runtime")
set(CPACK_COMPONENT_LIBRARIES_GROUP "Development")
set(CPACK_COMPONENT_HEADERS_GROUP "Development")

set(CPACK_COMPONENT_GROUP_DEVELOPMENT_DESCRIPTION
   "All of the tools you'll ever need to develop software")

# Define NSIS installation types
set(CPACK_ALL_INSTALL_TYPES Full Developer)
set(CPACK_COMPONENT_LIBRARIES_INSTALL_TYPES Developer Full)
set(CPACK_COMPONENT_HEADERS_INSTALL_TYPES Developer Full)
set(CPACK_COMPONENT_APPLICATIONS_INSTALL_TYPES Full)
 
# Must be after the last CPACK macros
include(CPack)

Generate the Visual Studio solution

  1. Open up a Command Prompt. Change to the build directory.

    D:> cd \temp\ComponentExampleStart\Source\build
  2. Use the cmake command to generate the build files.

    D:> cmake -G "Visual Studio 14 2015 Win64" ..

    The build files are generated.
Build the Visual Studio solution and generate the installer
  1. Using Visual Studio, open up the generated solution e.g. MyLib.sln.

  2. Since I wanted a release build, in Visual Studio I changed from Debug to Release, as shown below. Then I selected Build.

    The release binaries are generated.
  3. Next, in the Solution Explorer, select and mouse right click on the PACKAGE project.

    A pop down menu appears.
  4. Choose Build.



    CPack processing message appears.


    The NSIS installer is generated.


Monday, September 12, 2016

C++ example code for reading an Sqlite database from multiple threads

SQLite works well with threads as long as a single thread has its own database connection, or so they say in the documentation. I wanted to make a lot of database read only select calls on a table from multiple threads and I coded my C++ program according to the guidelines. But I found out some of the threads will get a "database is locked" exception after a while, and any subsequent calls to the database will raise a "library routine called out of sequence" exception. Changing the SQL connection to other types such as SQLITE_OPEN_FULLMUTEX, SQLITE_OPEN_NOMUTEX, and or permutations did not solve my problem.

Eventually I decided on a workaround to give each thread an in-memory copy of the database. The following code snippets show a simple example. The example uses the light weight C++ SQLite wrapper from https://github.com/aminroosta/sqlite_modern_cpp

The runThread function creates a copy of the original source database example.sqlite and puts it in memory with a unique connection string. Subsequently, it performs some read operations on the in-memory database.
1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
int runThread(int threadNumber) {
 try {
  cout << "[" << threadNumber << "]" << "Running" << endl;
  stringstream ss;

  // Form the connection string to the in-memory database
  ss << "file:memory" << threadNumber << "?mode=memory";
  string inMemoryDbConnectString = ss.str();
  ss.str("");

  // Open a database connection to the in-memory database
  database inMemoryDb(inMemoryDbConnectString);

  // Open a database connection to the original source database
  database sourceDb("example.sqlite");

  // Copy the source database to memory
  auto sourceConnection = sourceDb.connection();
  auto state = 
   std::unique_ptr<sqlite3_backup, decltype(&sqlite3_backup_finish)>(
    sqlite3_backup_init(inMemoryDb.connection().get(), "main", sourceConnection.get(), "main"),
    sqlite3_backup_finish
    );
  if (state) {
   int rc;
   // Each iteration of this loop copies 500 database pages from database db to the backup database.
   do {
    rc = sqlite3_backup_step(state.get(), 500);
//std::cout << "Remaining " << sqlite3_backup_remaining(state.get()) << "/" << sqlite3_backup_pagecount(state.get()) << "\n";
   } while (rc == SQLITE_OK || rc == SQLITE_BUSY || rc == SQLITE_LOCKED);

   // Do some reading from the in-memory database table
   for (int i = 0; i < 10000; i++) {

    double seconds = 452220.0 + (double)threadNumber + (double)i;
    double lower = seconds - 1;
    double upper = seconds + 1;

    ss
     << "SELECT seconds, x, y, z FROM attendance WHERE seconds >="
     << lower
     << " AND seconds <="
     << upper
     << " ORDER BY seconds ASC";

    inMemoryDb << ss.str() >> [&](
     double seconds, double x, double y, double z) {

     // do something with the returned records

    };
    ss.str("");
   }
  }
 }
 catch (sqlite_exception &e) {
  cout << "[" << threadNumber << "]" << "Error:" << e.what() << endl;
 }
 cout << "[" << threadNumber << "]" << "End" << endl;

 return 0;
}


Finally, the main function that creates multiple threads and waits for them to complete.

1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
int main(int argc, char **argv)
{
 // Create threads and put them into an array
 thread threads[3];
 for (int i = 0; i < 3; i++) {
  threads[i] = thread(runThread, i);
 }

 // Wait for the threads to complete
 for (int i = 0; i < 3; i++) {
  threads[i].join();
 }

 return 0;
}

An screenshot of the program in execution. Since each thread has a copy of the database, the database is never locked by other threads and I will never encounter the "database is locked" exception.

Note that this workaround will not work if read and write operations are required.