François-René Rideau (fare) wrote,
François-René Rideau

  • Music:

Dynamic software development

Here are a few things I'd like to say about dynamic software development, after I've just had the opportunity to test first hand with CTO what I had been studying in theory and through the experiences of other people.

CTO is nicely layered on top of the official CLiki and Araneida by dynamically redefining those functions that we need to change. This is very neat, and allows to function without having to even ever touch the original source. The same trick can also be used at the command line, while the server is running, or while in the debugger, so that modifications are usually tried and debugged on a live server before they get committed. Global initialization can also be tested by running a new process up to the point where it fails to bind the already taken server's address -- don't forget to do that (stupid me did, once). Dynamic languages are really great for maintaining long-lived dynamic sites.

There are limitations to the dynamicity of LISP, however. First, packages can be a mess, and it is clumsy and unobvious to fix a misdefined package structure, uninterning symbols that shouldn't be there, exporting and shadowing others, etc.; Joe Marshall has written about that (gotta find the URL).

Then, the standard doesn't seem to allow portable redefinition of generic-functions, so I had to be clever about defining new ones and somehow leaving a stub where the old ones were (so in a sense, it's still possible to redefine them, but in another sense, it's a kluge that works as a mostly static fix but not as a truly dynamic solution).

Finally, redefining macros is easy enough, but propagating the changes (if needed) must be done manually, which can be very painful; I didn't need that for CLiki, and somehow it's better not to always propagate automatically in a blind way, least you fall into strange loops, but it makes you feel that CL development environments are not so good as they ought to be if there hadn't been that Lisp winter (maybe SLIME will bring a new breath into good LISP development environments). Lisp lets you build your own domain-specific languages as adapted to your needs, but doesn't provide specific tools for robust vertical integration, yet: so very far ahead of competition, it still has a long way to go.

Another limitation of Common Lisp is its lack of a proper concurrency model like that of Erlang. That would have helped tremendously, says, at (1) making indexing asynchronous, yielding faster response times (2) making transaction management more robust, avoiding stubborn maintenance (3) making schema evolution more seamless and maybe even incremental, with the new version talking to the old one so as to migrate it with two versions running at once. (4) simplifying some of the logic (5) allowing for user code to run in protected ways in a controlled subthread thread, etc. sbcl-mt does have threads, but they are not meant to provide for the robust massive-parallel concurrency that makes it natural to program with concurrent agents.

As for typing, I've also been bitten by one or two stupid bugs that a static type analyzer would have found immediately, and that took me a tad bit longer to track with dynamic tools. Also, fixing things dynamically doesn't guarantee that you haven't messed up anything in the propagation of your dynamically static assumptions to less-used parts of code -- there too static typing could help. However, unless and until static typing can help with schema migration, there is no chance that it can allow to dynamically patch a live system with live data, even less so with persistent data. Just where are update-instance-for-different-class and update-instance-for-redefined-class in OCaml? Upgrading types means change of typing not at the individual object level, but as an atomic commit of many changes at the system level: a static type system then ought to build functors that modify the state of the whole computation, with first class computing systems! Also, in a static system suited to dynamic software evolution, placeholders for future typed computations will have to be replace the use of NIL and similar "empty" or "not implemented yet" tags that typically serve in dynamic systems; requiring the programmer to actually fill slots can do more harm than good. (and this may happen at the level of types, too!)

All in all, dynamic "the developer interacts with a computer system that evolves, each bringing his specific proficiencies in a mutual cooperation" development is fun, and is much more in tune with my cybernetic way of thinking than static "the developer is an all-knowing god who nevertheless makes an awful lot of mistakes at building applications that restart from scratch at every change" development. And it's fun even though this paradigm currently suffers from lack of stable featureful integrated infrastructure -- but let's call that an opportunity. Now, I'm looking ahead to using something like AP5, with data prevalence from bknr and modal interfaces from Uncommon Web or even tricks like CLIM and CWEST, to build great dynamic lisp applications. And the future will be filled with Lisp Machines once again...

Remember: programming is but debugging the empty program.

Tags: code, code evolution, dynamism, en, hacker, lisp, quotes, smop, tao of programming, tunes
  • Post a new comment


    Anonymous comments are disabled in this journal

    default userpic

    Your reply will be screened

    Your IP address will be recorded