My first experience of travis-ci with DBD::ODBC and repercussions

I've heard people mention travis-ci but not really paid much attention to it, that is, until yesterday when I issued a pull request from my github account for a few minor fixes to DBI. I wondered if my pull request was applied and took a quick look at DBI on github to see my pull request was still pending. Clicking on my pull request I see a "All is well — The Travis CI build passed" and just clicked on it. The details from travis-ci showed me that once my change was applied to DBI the DBI test suite was run on 4 versions of Perl and it passed all of them. This seems a wonderful resource so I thought, I'd like that on DBD::ODBC too.

So I took a look at the service hooks on github for DBD::ODBC and just followed the instructions. 1) login to travis-ci with your github account and let travis-ci see your github account 2) enable DBD::ODBC for travis-ci 3) get your token from your travis-ci profile and plumb it into the travis-ci service hook and enable it, 4) click the test hook (which said the test payload was sent but I didn't see any result of it doing so). Then I copied DBI's .travis.yml file, committed it and sat back waiting for travis-ci to do its thing. It failed with "The command "cpanm --quiet --installdeps --notest ." failed. Retrying, 2 of 3." see /home/travis/.cpanm/work/1380285065.1208/build.log for details". How the hell do you view that file?

So whilst I was working that out I realised the VM travis-ci is running my test on probably doesn't have the unixodbc and unixodbc-dev packages installed. Easy, just add:

 - sudo apt-get install -qq unixodbc unixodbc-dev

to my .travis.yml file. Once this was committed travis-ci ran again and now I can see the build of DBD::ODBC and the test suite results. Basically, most of the tests did not run because there was no database to connect to but the kwalitee test failed because my FAQ, TO_DO and Changes pod files don't have "use strict" in them :-( (even though there is no Perl code in those files!). So how did I not see the kwalitee test fail in my own environment? Before I got the real answer I took a quick look at the test and it is skipped unless there is a .git dir present or $ENV{IS_MAINTAINER}. Then I realised I switched perls and forgotten to install Test::Kwalitee. I probably shouldn't have but to quieten it for now I added a "use strict" to those pod files and then wondered how I was going to get an ODBC driver on the travis-ci system.

On #dbi Michiel Beijen pointed me at Databases and other services in the Travis CI Environment and I saw SQLite so thought that would be an easy choice for an ODBC driver. However, I've not run the DBD::ODBC test suite against the SQLite ODBC driver for ages (there are just too many ODBC Drivers to continually test) so I ran the test suite in my own environment and found quite a lot of failures. To be honest, this is no great surprise. The ODBC API reference is a book around 2 inches thick and few drivers pass all the tests anyway (including some MS ones).

One test failure was down to a new test I'd added for DBI's table_info method when someone reported a bug in DBD::ODBC to me. The failure was down to the fact SQLite does not support Schemas and Catalogs and the test assumed all databases did - fairly easily fixed with a few calls to DBI's get_info method to check check support for schemas and catalogs (thankfully the SQLite ODBC Driver authors had filled in SQLGetInfo correctly).

The second problem turned out to be a segfault in a fairly recent test to check the new odbc_get_diagrec methods I added to DBD::ODBC. This turned out to be a lot harder to find and ended up as a bug in unixODBC which was fixed BUT not released yet and in any case, not available to travis-ci even if it was because it looks like travis-ci uses debian and their packages are a bit behind. I put this problem on the shelf for a while.

The third failure was just a straight forward bug in the SQLite ODBC Driver so I set off to report it but no one seems to know how/where you report bugs in the SQLite ODBC Driver and I could not find a repository either. I mailed the email address at the bottom of the SQLite ODBC Driver page and Christian replied saying there isn't a bug tracker or public repository so just send him the issues and/or patches (which I've not done yet but I will).

The fourth problem was in the native execute_array test which is a hard test for any ODBC driver to pass as it requires support for arrays of parameters and the reporting of errors per row. I'll report this to Christian too but even if it is fixed as far as I can see I won't have access to either a new SQLite ODBC Driver on travis-ci (the current debian package is quite old) and the same applies to unixODBC. So, to get some test coverage on travis-ci I'll need to skip a few tests but I only want to do that when running on travis-ci. How does the test suite know it is running under travis-ci? A question I've yet to answer.

I'm also told I should be getting emails on travis-ci test failures but I'm not seeing those as yet - probably some setup I've not yet done?

So, an interesting few hours and I'll be working to get travis-ci working well with DBD::ODBC as I think this is a very useful resource.