DENDROCHRONOLOGY

I have become fascinated with dendrochronology (dendro). Dendro is the science that analyzes temporal and spatial consistencies of various processes using tree rings. The tree rings are dated to their exact year of formation. This data can be used across several disciplines: anthropology, geology, climatology, and ecology– to just name a few.

Of course, there is a ton of data to analyze. The primary data bank is located at The International Tree-Ring Data Bank and is maintained by the NOAA Paleoclimatology Program and World Data Center for Paleoclimatology. It includes raw ring width or wood density measurements, and growth indices for a site.

I am very excited to combine this mass amount of raw data with the power of lisp! I will document my adventures for those who are interested.

It has been awhile

I have been swamped with writing code for an OCR/Scanning system for my employer and being a team leader for 14 people (10 developers). Haven’t had too much time to continue Lisp development.

Recently, I had to do quarterly reviews and gave as an objective that each developer should learn a new language over the course of the next year.Each person was to choose a language they did not know and that would, hopefully, be mind expanding for them.

Most of the people on my team will be learning Python. One developer chose Erlang, though. Three others, including me, chose Haskell. No one chose Lisp or Scheme. Sad. The reasoning for Erlang and Haskell had to mainly do with wanting to learn a functional programming language that promised to be able to take advantage of the coming concurrency revolution through multiple cores.

Lisp was viewed as too old and having horrible syntax. I personally find it elegant and still far ahead of it’s time– although, I am afraid that the “ahead of it’s time” mantra is causing a lot of the community to rest on their laurels instead of pushing into the future.

I will be releasing some updates for trivial-freeimage in the upcoming weeks as well as a wrapper for the ocr engine TOCR. This little engine has outstanding character accuracy and is cheap– 40$ for the standard version and 86$ for the Euro version.

Depressing

I recently ventured into the Lisp community with a trivial wrapping of a library called FreeImage. I used CFFI for this and was quite pleased– well, except for one thing. That thing would be that I had to specially build the library to not use stdcall. CFFI does not support stdcall, yet (if ever). I generally prefer being able to build libraries without changing the source for a single language. So, I could just post instructions on what is needed to be done to the FreeImage sourcecode and let the user still get the source from the FreeImage web-site or I could always do a build and post it with trivial-freeimage. I have even thought about asking the developers of FreeImage if they would except a patch that would allow a user to use a define of LISP or some such thing to build especially for cffi. I would rather do none of the above. I suppose I will need to create a backend for each Lisp I wish to support– using their respective ffi– and then maybe have a front-end that hides it all away from the end-user of trivial-freeimage. That is rather disappointing as I very much like cffi. I wish cffi had support for stdcall and cdecl like ctypes in python (just found that out today from a co-worker who thought my situation was amusing and wanted to show off ctypes).

update:
The cffi developers are looking into adding support for stdcall. I have offered to help in anyway that I can. I am looking at the source of cffi to try and grok it. I did not mean to come across as one of words and not action. I also did not mean to come across as not appreciating cffi as I do appreciate it very much. CFFI does not have a donate paypal option but if they will institute one I will donate to help out since it will be unlikely I will be up to speed on the source before they finish adding stdcall.

Portable Graphical Widgets

Recently, I was advocating using Lisp at work for some utility apps when a co-worker claimed that we should use Java because we needed true cross-platform capabilities. We went back and forth on this for several minutes and I could tell he thought he had me when he brought up that they would need to have GUI’s. We have to have the ability to run our apps on Windows as well as Linux. So, enter wxcl. I think I may be on the way to converting another Java guy…..

update for trivial-freeimage

I finished making trivial-freeimage loadable with asdf. I also wrote a macro called with-dib (much thanks to zach for suggesting this).

(asdf:oos 'asdf:load-op :trivial-freeimage)
(use-package :trivial-freeimage)

(with-dib (dib "24.tif")
   (list (get-height dib)
         (get-width dib)))

with-dib will load an image for you and unloads it as well. It uses unwind-protect to ensure the unloading of the image in case of error.

trivial-freeimage

It’s been a slow start to the new year; but, here is a new package called trivial-freeimage. It wraps most of the FreeImage Library. I got this to work on Windows with CFFI 0.9.2 by building the FreeImage dll without stdcall linking. It was a simple edit to the header file. Anyway, here is a simple example that has functions to find the top and bottom of an image of text from a tiff-g4 scanned page:

(require 'asdf)
(require 'cffi)
(load "trivial-freeimage.lisp")
(in-package :trivial-freeimage)

(initialize 0)
;load a tiff-g4 image and get it's height and width
;width is divided by 8 due to 1 pixel per bit in tiff-g4 format

(defvar dib (load-image tiff "24.tif" 0))
(defvar height (get-height dib))
(defvar width (/ (get-width dib) 8))

;as soon as we find a black pixel notify
(defun row-has-black? (row)
	(loop for i from 0 upto (- width 1) do
		(if (> (mem-aref row :uchar i) 0)
			(return t))))

;find the top of the frame of text
(defun find-top (dib)
	(loop for row from (- height 1) downto 0 do
		(if (row-has-black? (get-scan-line dib row))
			(return (- height row)))))

;find the bottom of the frame
(defun find-bottom (dib)
	(loop for row from 0 upto height do
		(if (row-has-black? (get-scan-line dib row))
			(return (- height row)))))

;always unload your dib's or you will never free the memory
(unload-image dib)
(de-initialize)

Also, note that FreeImage stores DIB’s with the origin being in the bottom left.

One other example– to convert a tiff file to png:

(in-package :trivial-freeimage)
(initialize 0)
(defvar dib (load-image tiff "image.tif" 0))
(save-image png dib "image.png" 0)
(unload-image dib)
(unload-image dib2)
(de-initialize)

I will be posting more examples while I work on the documentation of how to use the wrapping. Also, I will continue to add more of FreeImage’s API to the wrapping. I plan on adding a higher package for image editing that will make use of trivial-freeimage.

Just another update

I have had a few people ask for links, so here they are:

FreeImage
openCV

Just an update

Today I am working on making a wrapper for freeimage that is nicer than just the trivial cffi wrapping I had already done for it. I am also looking at doing the same for opencv. Although, I guess I could release them as trivials– not sure what the preference in the community is in regards to this. If anyone has an opinion on whether a trivial release would be useful for others to build their own packages on or not please drop a comment.

CLORB: CORBA for Lisp

Well, I thought about it for awhile and decided I would post something about CLORB in particular. Where I work we use CORBA as the transport for our grids. I have always wanted to be able to use Lisp as a client– yay! CLORB. Now, I know there are other CORBA implementations for Lisp– but this one is free. That last word there– free– is very, very important when you are trying to establish a foothold for Lisp in the corporate IT world. I use SBCL on Linux and CLISP on Windows because they are free. That means I don’t have to go through the entire process of requesting funds for LispWorks or ACL. Which, also means, I don’t have to justify an expense for something no one else in the company for which I work is using. So, now that I have free Lisp tools and CLORB I can interoperate with all of the other services that have been developed in other languages at the company I work. The interesting thing about this use of free tools is that once I have a few projects/services in place using Lisp I can justify actually buying LispWorks or ACL if needed. Now back to CLORB– I was very happy at how easy it was to get a client up and talking to a python service. I am using OmniNames as my nameservice. The following snippet will enable the connection and use of a corba service– in this case the python corba service provides a single method called getsyphs.

(require :asdf)
(require :sb-bsd-sockets)
(require :clorb)

(defvar *orb*
    (CORBA:ORB_init
         (list "-ORBInitRef" "NameService=corbaloc::10.85.90.144:2809/NameService")))

(corba:idl "syph.idl")

(defvar *obj*
        (op:resolve_initial_references *orb* "NameService"))

(defvar *rc*
        (op:narrow 'cosnaming:namingcontextext *obj*))

(defvar syph
        (op:resolve "BCS.syph/BCSSyph.Object"))

(defvar rt
        (op:narrow 'BCS:Syph syph))

(op:getsyphs rt "014600AIR CANADA")

The only downside I have experienced with CLORB is that it depends on cpp to preprocess idl. This is a small problem if you are on Windows and don’t have a cpp. I got around it by having CLORB generate all of the Lisp code to a single file for the idl on Linux.

(CORBA:IDL "syph.idl"  utput "syph.lisp")

Then in CLISP on my Windows box I replaced

(corba:idl "syph.idl")

with

(load "syph.lisp")

and all was happy.

good-bye 2006

Well, another year is coming to a close. This has been a good year for me in terms of programming Lisp professionally. I became a Team Leader for a group of developers and we were charged with developing a new ocr system. We used several languages– right tool for the job mentality. My boss was so pleased with what we accomplished and the time-frame in which it took that I have now been given the task to design and replace a large legacy system. All of this success has been because I do not believe in one language for every task. I firmly believe in the right tool for the job. I also believe in rapid-prototyping. Following these beliefs has put me in the position that I can use Lisp in an enterprise I.T. setting. What have I been using Lisp for? I can’t give too many details at this point– but I can tell what packages I have been using or have created. I have wrapped FreeImage and OpenCV in the last month– I’ll be releasing them in the next month. I have also recently started looking at CLORB. I have gotten CLORB clients talking to omniORB python servents. I am also using wxcl. Right now I am working on a Name Utility that will list CORBA bindings in a wxcl gui. I will post the code on this site in about a week. Have a Happy New Year!