InfoQ announced the porting of SQLite to .NET two weeks ago. Because the community showed special interest in this project, we interviewed Noah Hart, the developer who did it, to find out how SQLite was re-written in C#.
Why did you do it?
My original goal was to deploy an application with an embedded database without additional dlls. After some searching, I found SQLite and decided to see if I could use part of the database engine code in a freeware application, Punjabi Kosh, that I had written. My first pass was a rewrite in VB, but I decided that it was not a good fit. I was wanting to rewrite Kosh and at the same time, wanting to learn C#.
How did you approach the porting process? Did you use a tool or was it a manual operation?
Step 1: Created a new C# project with an empty main routine, and added in all of the SQLite source and header files; This of course generated a plethora of error messages
Step 2: Commented out the entire C code. My thinking was to do incremental changes, and I needed to create a project that would compile.
Step 3: Since I was not familiar with C#, my next pass was to determine what the differences were between the 2 languages. Not a big surprise, but most of the syntax changes were minor. I was able to program most of them into Visual Studio macros to automate replacements. These were things like:
C C# -> . & ref assert Debug.Assert ==0 ==null #ifdef #if #ifndef #if ! Step 4: Decided to wrap the entire C code into a single sqlite class. In C, definitions flow from file to file, and the source is compiled in sequence. However in C#, each source files stand alone, so by making each file part of a partial class, I was able to access routines between files.
Step 5: C supports inline compiler macros, C# does not. I needed to change most of the #DEFINE into small routines or constants.
Step 5a: Make sure project can still compile.Step 6: Then the hard work started. I changed all the structs into objects, with public members
Step 6a: Make sure project can still compile.Step 7: Started uncommenting routines. This is where things started getting interesting. I learned about many many subtle differences such as how definitions and null pointers are handled; lack of union in C#; value vs. reference objects; behavior changes in switch, case; byte arrays and character strings address indexing. Also, many “helper” functions like atoi, printf, memcpy, strcmp, etc. were needed. In most cases I simply emulated them, rather then rewriting the code. The idea was to save conversion time, make sure the application worked, then start rewriting as C# centric code.
Step 7a: Make sure project can still compile.Then it was just
Step 8: while (not_done) {Step 7; Step 7a;}
Hart considers C#-SQLite a C emulation in C# rather than a port:
C# is object oriented, C is not. So it is really a misnomer to call what I’ve done a port. It is really more of a C emulation in C#. Most of the code still flows in C style, I’m using very little of Object methodology or C#-centric features.
The whole porting process took a bit over two years, all the work being done part time as a hobby. Hart started with 106,700 lines of C code and came up with 117,329 lines of C#, but
this is not a fair comparison, since in many places I’ve kept the original C code as comments for reference.
How would you appreciate the overall porting process? Painful? Fun?
It was a real learning experience. And my goal was also to learn how SQLite worked. I enjoyed getting into the internals of the application.
What lessons would you like to share about porting from C to C#?
Decide what parts you don’t need to port.
Always remember what your goal is.
Automate as much as possible.
Setup your process to allow incremental porting.
Watch your pointers.
Ask questions, really understand the answers.
There are about 30,000+ tests that C#-SQLite is already passing. Are those SQLite standard tests or some tests you created especially for this project?
These are the standard tests in the test harness that sqlite.org provides. From http://sqlite.org/faq.html:
(17) Quality assurance in SQLite is done using full-coverage testing, not by compiler warnings or other static code analysis tools. In other words, we verify that SQLite actually gets the correct answer, not that it merely satisfies stylistic constraints. Over two-thirds of the SQLite code base is devoted purely to testing. The SQLite test suite runs many thousands of separate test cases and many of those test cases are parameterized so that hundreds of thousands of tests involving millions of SQL statements are run and evaluated for correctness prior to every release.
However, all these tests ran in TCL, so I needed to port TCL into C# as well. I found a version of TCL that was already ported into Java, and from there I ported it into C#.
Side note, that was a lot of work!
How many tests are still not passing? How much work is required to make them all pass?
9. The work is determining if the test is even appropriate in the C# world. For example, some tests are related to big-endian vs. little-endian testing, which are not needed under C#.
What are the greatest benefits of using C#-SQLite compared with SQLite wrapper/adapters for .NET?
Many of the SQLite wrappers/adapters for .NET are wonderful. I would not consider C#-SQLite as a replacement for them.
What projects are going to mostly benefit from C#-SQLite? What would you use it for?
Embedding the SQLite engine in an application without need for external dlls, being able to use it in fully managed code with medium trust.
What are your plans for the future (related to C#-SQLite, of course)?
Removing the remaining p/Invokes and get it working under Silverlight.
Do you intend to continue porting future versions of SQLite?
Yes, ver 3.6.17 is now available.
How do you envision supporting this new project (referring to bugs, enhancements)?
I’ve setup a website http://code.google.com/p/csharp-sqlite/ and a discussion group http://groups.google.com/group/csharp-sqlite.
Do you need help from the community?
Yes. I am not an accomplished C# programmer by any stretch of the imagination. I am positive that the performance and C Sharpification of C#-SQLite would benefit from the input of additional developers.
Noah Hart is a developer interested in machine translation from English to Punjabi.