Monotone

You may remember that I am looking for a solution to the SNAIL problem, which will have to be based on epistemic monotonic logic. Well, monotone offers half of the solution, monotonic logic. And that is also the more useful half to me, considering that the instantaneous latency of my SNAIL networks is actually low: when my computers are connected to each other, then the communication latency between them is below a few tenths of seconds at most.

monotone is relatively immature, and I had problems with it; moreover, I don't expect it to ever provide the epistemic monotonic logic that I'm longing for. Nevertheless, its basic design seems like the Right Thing to me, the implementation seems sound, the resulting package is already very useful and very easy to get going with, the transition from CVS is remarkably painless, and I expect it to stabilize into something reliable and usable over time. Even in its current state, it is vastly superior to centralized systems such as cvs or its modern replacement subversion; it offers builtin support for importing from a CVS server and for synchronizing with ongoing CVS development. Incidentally, my central CVS server being offline for a week was what prompted me to finally switch. It is also more to my liking than the popular decentralized system darcs, since it maintains databases of everything that is (locally) known about all the code you want, instead of splitting it into as many active per-project, per-branch copies. A monotone checkout is thus very lightweight and very easy to tweak manually, yet robustly managed using cryptographic identification of file contents. File renaming is well-supported, and commits are of course atomic.

Still, I had a few annoyances with the current version of monotone, and I'd like to share my experience with you, so you may avoid a few pitfalls and get more productive.

Firstly, monotone was mightily confused by the entry for "." that it itself created in .mt-attrs when I did a monotone add . -- it was easy to fix this first problem by editing .mt-attrs by hand, but it took me quite some time to understand what was going on; happily, monotone itself could tell it was a bug, so I could choose the right tool to debug, namely strace. Now, I do monotone add $(monotone list unknown) to recursively add files below the top (or current) directory.

Another annoyance is that it insists in using its own network protocol for database synchronization, whereas it is not always possible or affordable to reserve a port for this service, even allegedly cryptographically secure: this involves issues of trust and wasted time by the administrators who manage the firewall and the reliable allocation of ports to servers. I solved this problem with a combination of ssh port redirection and zsh hackery.

# cat zsh.functions.monotone
mo_ad () { monotone add $(monotone list unknown) }
mo_rm () { \rm $@ ; monotone rm $@ }
mo_mv () { \mv $@ ; monotone mv $@ }
mo_di () { monotone diff $@ }
mo_cm () { monotone commit $@ }
mo_diff () { ( cd $1 && shift && mo_di $@ ) | less }
mo_commit () {( cd $1 && shift && mo_cm $@ )}
mo_up () {( cd $1 && shift && monotone update $@ )}

mo_run_server () {
  coproc ( exec monotone --quiet >& /dev/null \
                --db=$DB serve localhost:$PORT "$BRANCHES"
  ) ; SERVPID=$!
  echo "Launched a local monotone server with PID=$SERVPID"
  echo "for database $DB on port $PORT (locally)."
  mo_kill_server () {
    echo "Killing monotone server with PID=$SERVPID"
    kill $SERVPID
  }
  if [ -n "$MONOTONE_PASSWORD" ] ; then
    print -rp -- "$MONOTONE_PASSWORD"
  fi
}
mo_run_client () {
  echo "Connecting to host $HOST to run a monotone client"
  echo "for database $RDB on port $RPORT (remotely)."
  stty sane
  REMOTE_MONOTONE=(monotone --db=$RDB sync localhost:$RPORT $BRANCHES)
  SSH_ARGS=(
    -R ${RPORT}:localhost:$PORT
    $HOST
    ${(qqq)REMOTE_MONOTONE} # stty sane ;
  )
  if [ -n "${REMOTE_MONOTONE_PASSWORD:=$MONOTONE_PASSWORD}" ] ; then
    print -r -- "$REMOTE_MONOTONE_PASSWORD" |
      if command_p cotty 2> /dev/null ; then
        cotty -1 ssh -t $SSH_ARGS ### Allow for nice output on TTY.
      else ssh $SSH_ARGS ; fi ### Ugly TTY output.
  else
    ssh -t $SSH_ARGS
  fi
}
mo_sync () {(
  DB=$1 HOST=$2 PORT=${3:-35253}
  RDB=${4:-$DB} RPORT=${5:-$PORT}
  BRANCHES=${6:-'*'}
  mo_run_server
  trap mo_kill_server EXIT
  # sleep 1 # Give the server some time to breath before startup
  mo_run_client
  mo_kill_server
  trap "echo done." EXIT
)}

Note however that said hackery assumes that you configured get_passphrase (see below) on the local end of the mo_sync command. Indeed, monotone's key management is lacking and that makes for not so good compromises between security and usability: not only does it lack an ssh-agent-like system to manage keys (or an interface to ssh-agent itself -- why reinvent incompatible wheels?), it also insists in keeping the secret key in the database itself rather than in a separately manageable secret file. Happily you can customize monotone with a dynamic language; unhappily this is yet another unremarkable crippled language that I had to learn, namely lua; lua hackers on irc.freenode.net were very helpful. Kudos! Here is my ~/etc/monotone/monotonerc. I'm sure you can do better with respect to key management, but it's too much for my expected diminishing returns.

-- -*- lua -*-
-- In your ~/.monotone/monotonerc, insert the following:
--   dofile(os.getenv("HOME").."/etc/monotone/monotonerc")
--
-- You may also insert a definition for your passphrase:
-- function get_passphrase(keypair_id)
--  return "secret passphrase"
-- end

function get_netsync_read_permitted (branch, identity)
  if (identity == "fare@tunes.org") then return true end
  return false
end

function get_netsync_write_permitted (identity)
  if (identity == "fare@tunes.org") then return true end
  return false
end

function fare_mkSet(s)
  local rv = {}
  for w in string.gfind(s, "%S+") do rv[w] = true end
  return rv
end

fare_bad_extensions = fare_mkSet[[
	o a la lo so  pyc pyo  lib fas fasl x86f
	depend deps
	out tmp swp cache
	ps eps pdf
	aux dvi log bbl blg toc brf haux lot lof idx 4ct 4tc idv lg xref
	gz bz2 deb tar zip
	bak orig rej
]] -- gif png jpg txt

fare_bad_names = fare_mkSet[[
	out.tex init_mzscheme
	core
	foo bar baz quux toto tata tutu titi
	FOO BAR BAZ QUUX TOTO TATA TUTU TITI
	.cvsignore
]]

function ignore_file(name)
   -- Don't ignore a few precious cached things
   -- Get it from the WWW -- if (name == "fare/texstuff/fare.eps") then return false end
   local sname = "/"..name
   local _, _, basename = string.find(sname, "/([^/]*)$")
   local _, _, extension = string.find(basename, "%.(%w+)$")
   if (fare_bad_extensions[extension]) then return true end
   if (fare_bad_names[basename]) then return true end
   if (string.find(basename, "%~$")) then return true end
   if (string.find(basename, "^%.%#")) then return true end
   if (string.find(sname, "/CVS/")) then return true end
   if (string.find(sname, "/.svn/")) then return true end
   if (string.find(sname, "/_darcs/")) then return true end
   return false
end

Finally, monotone being very young, I had to use debian unstable so that I could find recent decent versions of it compatible with the same current netsync protocol version 5 (latest being 0.22, compatibility requiring 0.20 at least) on both my i386 and arm architectures (yes, monotone runs just fine on my good old jornada 820!) There again, debian hackers on irc suggested to RTFM man apt_preferences, and so I now have the following:

# cat /etc/apt/apt.conf
APT::Default-Release "testing";
# cat /etc/apt/preferences
Package: common-lisp-controller clisp sbcl cl-* monotone user-mode-linux
Pin: release a=unstable
Pin-Priority: 1000
# apt-get update ; apt-get install monotone

Then, the CGI interface to viewing monotone is a python script that will only work with a recent mod_python in apache 2.0, so I couldn't install it on fare.tunes.org yet. But I will eventually, thanks to user-mode-linux. It is recommended that it should also use its own additional copy of the database for security concerns.

Which brings us to disk usage. monotone, lacking some kind of relevance logic to sort between facts to keep and facts that can be thrown away, practically requires a copy of all versions of all the managed software in every machine. This can be unwieldy on very large multi-gigabyte projects. Also, data seems to be compressed internally, but is then stored as text-encoded binary in the underlying database, which is a weird inefficiency the rationale of which is probably laziness in a time of cheap big disks. If you're a hacker, you may call that room for improvement.

More room for improvement, beside all that was discussed above could include a fact superseding convention in addition to the previously suggested relevance logic (to allow to "erase" sensitive data that you don't want there without having to dump, hack and rebuild the whole database, losing all previous cryptographic certificates), epistemic logic (for efficient synchronization over real SNAIL disconnected networks), a simple protocol for a centralized authority to provide human-readable release numbers (to replace the lacking $Id$ feature, incompatible with general decentralized use), etc. Come on, guys, it's a SMOP!

I hope these tips will help you install monotone and be productive, and maybe convince you to give a try to this fine piece of software (and maybe even hack it).

Update: the new and improved ssh synchronization function (above), now can handle monotone passwords stored in shell variables. Security issues may apply if a malicious user having access to the machine may read these variables by somehow inspecting the memory of your processes; but then again, if someone can do that, crypto isn't the weak link in your security, anyway.

• Mini Sitemap:

• Post to Journal
• Downloads

• LJ Gift Shop
• Safety Tips

• Support & FAQs
• Be a Support Volunteer

• Feedback & Contact Us
• Report Abuse

• Site News
• More... (Full Sitemap)