A small metadata adventure

My nephew Ben, his friend Chance, and I went on a backpacking trip in mid-July (7/12 through 16th), from Twin Lakes (northeast of Bridgeport) into Yosemite. Here are the pictures from the Twin Lakes trip:
http://picasaweb.google.com/dlandauer/IgLTwinLakes The actual trip report will be in another posting. [Linked here]

I had a small problem to solve once we got back: Ben had one of my cameras, and I had another. Each one can set the "creation time" in the JPEG file's metadata, but the two cameras' internal clocks were not synchronized, and I wanted to send them up to Picasa in chronological order. The rest of this weblog entry isn't really about the backpacking trip (read the pictures' captions for that, or the forthcoming trip report), but more about how I solved the time-synchronization problem.

[This turns out to be a problem others have run into, and there's a Linux program called "jhead" that can do this kind of date manipulation. But writing the small Python program was quick and somewhat entertaining...]

I did some bulk file renaming to make the filenames lowercase, remove some leading zeros, and make the camera names -- Lumix and Sony -- clearer. I run Mac OS X, which has a command called "sips" (Simple Image Processing System"?); I used it to extract the recorded times for each photo, and listed each time and its corresponding filename into a text file. After another couple of global substitutes, I had a Python array containing the filenames and their nominal times, like this:

pix_and_times = [
  [ 'lmx50553.jpg', '2009:07:10 07:06:45' ],
  [ 'lmx50555.jpg', '2009:07:11 12:15:30' ],
  ...
  [ 'lmx50795.jpg', '2009:07:16 16:08:07' ],
  [ 'lmx50796.jpg', '2009:07:16 16:50:09' ],

  [ 'sony6308.jpg', '2009:07:12 05:36:12' ],
  [ 'sony6309.jpg', '2009:07:12 05:36:22' ],
  ...
  [ 'sony6524.jpg', '2009:07:16 06:06:31' ],
  [ 'sony6528.jpg', '2009:07:16 06:07:48' ],
  [ 'sony6529.jpg', '2009:07:16 06:08:25' ]
]

The next question was, when and where did we take any pictures that I knew were from around the same time? What I came up with was that on our last hiking day, in Kerrick Meadow, we saw a pair of Mountain Chickadee chicks, running around on a muddy streambed, and I knew we had both taken pictures of them. So I found a couple of the corresponding photos

Lumix chick	Sony chick

and figured out that they were 12 hours and 23 minutes apart.

Armed with that knowledge, I wrote 50-some lines of python that renamed the files so that their filenames, sorted alphabetically, would also be in correct chronological order. The guts of the code:

  # Make an object for each photo
  pixobs = map(Pixob, pix_and_times)

  # Parse the dates of the photos, and adjust my notion of the creation
  # time for only the sony ones.
  for ob in pixobs:
    ob.parse_date()
    ob.adjust_sony_time()

  # Sort by adjusted date
  pixobs.sort( date_cmp )

  # Print out a "mv" command to rename each file with a name
  # that will sort the way we want.
  for nr, ob in enumerate(pixobs):
    print 'mv ' + ob.fn + ' ' + aatrans(nr) + ob.fn

The aatrans function translates an integer between 1 and 676 into 'aa', 'ab', 'ac', ... 'zx', 'zy', 'zz'

  def aatrans ( nr ) :
    qq = nr // 26
    rr = nr % 26
    return atoz[qq] + atoz[rr]

Finally, I ran the shell on that output, resulting in files with names like these interleaved ones, from somewhere in the middle:

  cvlmx50622.jpg
  cxsony6314.jpg
  cylmx50623.jpg
  czsony6315.jpg
  dalmx50624.jpg
  dblmx50625.jpg
  dcsony6316.jpg
  ddsony6317.jpg

(At some point, I'll figure out how to put the python source code somewhere on the web. Note that it doesn't include the bulk renames and sips data extraction.)

My favorite part of this is how well the time-alignment was demonstrated. After aligning the photos via the chicks, I went back and looked at this sequence of photos taken as we were climbing the snow and rocks towards Burro Pass. I took a photo of Ben, way up on the rocks above us, then I zoomed way in and took another one, nine seconds later. In between those two shots, the sorting program placed the picture that Ben took, of me taking one of those pictures! Here's that sequence:

Back to civilization

Just returned from a five-day backpacking trip mostly in Yosemite. Twin Lakes - Benson Lake loop, clockwise. Got some nice pix. Lots of snow on the north sides of Mule and Burro Passes. Lots and lots of bugs. Had the big Benson Lake Riviera beach all to ourselves for an entire morning. Kerrick Canyon and Meadow was my favorite part -- wide open flat meadow, interesting peaks and domes on both sides, and a pretty easy climb out at the top, via Peeler Lake. My nephew Ben did great -- he's definitely in his element out there, now that he's 14 and been on six or so trips (mostly shorter, though). His friend Chance was on his first backpacking trip ever, and his determination overcame his inexperience and low-ish fitness level, so by the last day he was walking down the last part with Ben, and I was dragging along behind. Pix will be along at some point.

Busy times

Planning some backpacking trips for the summer. With luck, I'll have some photos to show from it. Lots of stuff I can't really talk about going on at work. It's nice to see Scala getting some interest. I biked over to LinkedIn about a month ago, to attend a talk/panel with Martin Odersky, David Pollak (creator of Lift), and Nick Kallen from Twitter.

Updated Reminder about DST Asymmetry

It's March: time for my annual reminder about the asymmetry of daylight savings time. If you've seen this before, well, here's a reminder. This time, there's a bonus: a Google App Engine application that'll make a nice graph.

The summary: Daylight Savings Time is not symmetrical around the solar calendar — it does not "surround" the days of longest light as one might expect if one hasn't thought about it. So if you like to do outdoors stuff (e.g., hiking or mountain-bike riding) while it's light out, especially mid-week after work, the time to get started with that sort of outdoors stuff is NOW.

Some details: If we were to start DST at the spring equinox and end it at the autumn equinox, then it would be close to symmetric in the sense I'm talking about. Around the day we spring ahead into daylight savings time, sunset would change from being (say) 6:10 to being 7:10; and the day we fall back to winter time, it would go from 7:10 back to 6:10.
But we don't do the DST changes symmetrically around the equinoxes: we spring ahead in early March, roughly one or two weeks before the after the spring equinox, and we fall back in early November, around six or seven wo weeks after the fall equinox. This means that it's lighter later into each day at the start of DST than it is at the end. In fact, a little bit of trigonometry (or some actual observation) will tell you that the equinoxes are the times of the fastest change in the days' lengths, so it's lighter a lot later at the start of DST than at the end.

Around here (San Francisco Bay Area), the springtime change this year (2009) made our notion of sunset change from 6:10pm to 7:10pm. So it's already worth getting out for a short (60- to 90-minute) hike or ride after work even if you have to work until 6:00pm.

By contrast, the autumn change around here, this year, makes our label for "sunset-time" change from 6:10pm to 5:10pm. So that's a full hour's difference, compared to the springtime change. It generally means that despite the usually dry autumn weather we get, those of us who have to work until 6:00pm or later have few choices by October — learn to like riding with lights (and to find places where it's legal), or stick to riding or hiking only on weekends, "hookey" weekdays, or long "lunch"es.

Bottom line: For those of us for whom snow is not an issue, the dry parts of March or April are the time to start those after work hikes or rides.

So after years of writing the occasional exhortation/warning about this, I finally wrote an app engine app to display the sunrise and sunset data graphically, so you could more easily eyeball and internalize the asymmetry. Here's the link: DST Asymmetry. It's somewhat fragile at the moment, and still has a couple of extraneous debugging links and a css that, um, could be nicer.

Finally, here are links to a couple of earlier versions of this warning: Seven years ago; and Two years ago.

Daylight Saving Time Asymmetry Graphs

Apropos of my last posting, and jeez that was an awfully long time ago ..., I was playing around with Google's graph/chart in-URL API, and came up with these neat graphs that show how asymmetric daylight savings time is. Maybe someday I'll figure out how to put in vertical lines, too, to show where the equinoxes are. (Sorry, I've temporarily edited the blog's template to fix the screwups that were not permitting the graphs to show. It made the sidebar go all the way to the bottom. Will fix that one of these days.) [Later: Ok, I fixed that by importing my customized blogger template from another blogger blog, and further tweaking it to fit this one. Whee.]

With line at 5:30pm:

---

With line at 6pm: