Once is never

Image by ZEEVVEEZ on Flickr, licensed CC-BY. Ten points if you can tell what it is...

My eldest daughter is in grade 5, so she's getting into some fun things at school. This week the class paired off to meet a challenge: build a container to keep hot water hot. Cool!

The teams built their contraptions over the weekend, doubtless with varying degrees of rule interpretation (my daughter's involved HotHands hand warmers, which I would not have thought of), and the results were established with a side-by-side comparison. Someone (not my daughter) won. Kudos was achieved.

But this should not be the end of the exercise. So far, no-one has really learned anything. Stopping here is like grinding wheat but not making bread. Or making dough, but not baking it. Or baking it, but not making it into toast, buttering it, and covering it in Marmite...

Great, now I'm hungry.

The rest of the exercise

How could this experiment be improved?

For starters, there was a critical component missing: control. Adding a vacuum flask at one end, and an uninsulated beaker at the other would have set some useful benchmarks.

There was a piece missing from the end too: analysis. A teardown of the winning and losing efforts would have been quite instructive. Followed by a conversation about the relative merits of different insulators, say. I can even imagine building on the experience. How about a light introduction to thermodynamic theory, or a stab at simple numerical modeling? Or a design contest? Or a marketing plan?

But most important missing piece of all, the secret weapon of learning, is iteration. The crucial next step is to send the class off to do it again, better this time. The goal: to beat the best previous attempt, perhaps even to beat the vacuum flask. The reward: $20k in seed funding and a retail distribution deal. Or a house point for Griffindor.

Einmal ist keinmal, as they say in Germany: Once is never. What can you iterate today?

Introducing Striplog

Last week I mentioned we'd been working on a project called striplog. I told you it was "a new Python library for manipulating well data, especially irregularly sampled, interval-based, qualitative data like cuttings descriptions"... but that's all. I thought I'd tell you a bit more about it — why we built it, what it does, and how you can use it yourself.

The problem we were trying to solve

The project was conceived with the Nova Scotia Department of Energy, who had a lot of cuttings and core descriptions that they wanted to digitize, visualize, and archive. They also had some hand-drawn striplog images — similar to the one on the right — that needed to be digitized in the same way. So there were a few problems to solve:

• Read a striplog image and a legend, turn the striplog into tops, bases, and 'descriptions', and finally save the data to an archive-friendly LAS file.
• Parse natural language 'descriptions', converting them into structured data via an arbitrary lexicon. The lexicon determines how we interpret the words 'sandstone' or 'fine grained'.
• Plot striplogs with minimal effort, and keep plotting parameters separate from data. It should be easy to globally change the appearance of a particular lithology.
• Make all of this completely agnostic to the data type, so 'descriptions' might be almost anything you can think of: special core analyses, palaeontological datums, chronostratigraphic intervals...

The usual workaround, I mean solution, to this problem is to convert the descriptions into some sort of code, e.g. sandstone = 1, siltstone = 2, shale = 3, limestone = 4. Then you make a log, and plot it alongside your other curves or make your crossplots. But this is rather clunky, and if you lose the mapping, the log is useless. And we still have the other problems: reading images, parsing descriptions, plotting...

What we built

One of the project requirements was a Python library, so don't look for a pretty GUI or fancy web app. (This project took about 6 person-weeks; user interfaces take much longer to craft.) Our approach is always to try to cope with chaos, not fix it. So we tried to design something that would let the user bring whatever data they have: XLS, CSV, LAS, images.

The library has tools to, for example, read a bunch of cuttings descriptions (e.g. "Fine red sandstone with greenish shale flakes"), and convert them into Rocks — structured data with attributes like 'lithology' and 'colour', or whatever you like: 'species', 'sample number', 'seismic facies'. Then you can gather Rocks into Intervals (basically a list of one or more Rocks, with a top and base depth, height, or age). Then you can gather Intervals into a Striplog, which can, with the help of a Legend if you wish, plot itself or write itself to a CSV or LAS file.

The Striplog object has some useful features. For example, it's iterable in Python, so it's trivial to step over every unit and perform some query or analysis. Some tasks are built-in: Striplogs can summarize their own statistics, for example, and searching for 'sandstone' returns another Striplog object containing only those units matching the query.

  >>> striplog.find('sandstone')
  Striplog(4 Intervals, start=230.328820116, stop=255.435203095)

We can also do a reverse lookup, and see what's at some arbitrary depth:

  >>> striplog.depth(260).primary  # 'primary' gives the first component
  Rock("colour":"grey", "lithology":"siltstone")

You can read more in the documentation. And here's Striplog in a picture:

View fullsize

An attempt to represent striplog's objects, more or less arranged according to a workflow.

Where to get it

For the time being, the tool is only available as a Python library, for you to use on the command line, or in IPython Notebooks (follow along here). You can install striplog very easily:

  pip install striplog

Or you can clone the repo on GitHub. 

As a new project, it has some rough edges. In particular, the Well object is rather rough. The natural language processing could be much more sophisticated. The plotting could be cuter. If and when we unearth more use cases, we'll be hacking some more on it. In the meantime, we would welcome code or docs contributions of any kind, of course.

And if you think you have a use for it, give us a call. We'd love to help.

---

Postscript

I think it's awesome that the government reached out to a small, Nova Scotia-based company to do this work, keeping tax dollars in the province. But even more impressive is that they had the conviction not only to allow allow but even to encourage us to open source it. This is exactly how it should be. In contrast, I was contacted recently by a company that is building a commercial plug-in for Petrel. They had received funding from the federal government to do this. I find this... odd.

The perfect storm

Since starting Agile late in 2010, I have never not been busy. Like everyone else... there's always a lot going on. But March was unusual. Spinning plates started wobbling. One or three fell. One of those that fell was the blog. (Why is it always your favourite plate that smashes?)

But I'm back, feeling somewhat refreshed after my accidental quadrennial sabbatical and large amounts of Easter chocolate. And I thought a cathartic way to return might be to share with you what I've been up to.

Writing code for other people

We've always written code to support our consulting practice. We've written seismic facies algorithms, document transformation routines (for AAPG Wiki), seismic acquisition tools, and dozens of other things besides. But until January we'd never been contracted to build software as an end in itself.

Unfortunately for my sanity, the projects had to be finished by the end of March. The usual end-of-project crunch came along, as we tried to add features, fix bugs, remove cruft, and compile documentation without breaking anything. And we just about survived it, thanks to a lot of help from long-time Agile contributor, Ben Bougher. One of the products was striplog, a new Python library for manipulating well data, especially irregularly sampled, interval-based, qualitative data like cuttings descriptions. With some care and feeding, I think it might be really useful one day.

The HUB is moving

Alongside the fun with geoscience, we're in the midst of a fairly huge renovation. As you may know, I co-founded The HUB South Shore in my town in 2013. It's where I do my Agile work, day-to-day. It's been growing steadily and last year we ran out of space to accept new members. So we're moving down to the Main Street in Mahone Bay, right under the town's only pub. It's a great space, but it turns out that painting a 200 m² warehouse takes absolutely ages. Luckily, painting is easy for geologists, since it's basically just a lot of arm-waving. Anyway, that's where I'm spending my free time these days. [Pics.]

MAder's Wharf, by the frozen ocean.

The ship's knees

Co-founder Dave painting trim

Shovelling snow

What my house has looked like for the last 8 weeks.

Seriously, it just will. Not. Stop. It's snowing now, for goodness sake. I'm pretty sure we have glaciers.

What does this have to do with work? Well, we're not talking about Calgary-style pixie dust here. We ain't nipping out with the shovel for a few minutes of peaceful exercise. We're talking about 90 minutes of the hardest workout you've ever endured, pointlessly pushing wet snow around because you ran out of places to put it three weeks ago. At the end, when you've finished and/or given up, Jack Frost tosses a silver coin to see if your reward will be a hot shower and a course of physiotherapy, or sudden cardiac arrest and a ride in the air ambulance.

Events

There is lots of good techno-geophysics to look forward to. We're running the Geoscience Hackathon in Calgary at the beginning of May. You can sign up here... If you're not sure, sign up anyway: I guarantee you'll have fun. There's a bootcamp too, if you're just starting out or want some tips for hacking geophysics. Thank you to our awesome sponsors:

There's also the geophysics mini-symposium at SciPy in Austin in July (deadline approaching!). That should be fun. And I'm hoping the hackathon right before SEG in New Orleans will be even more epic than last year's event. The theme: Games.

Evan is out there somewhere

Normally when things at Agile World Headquarters get crazy, we can adapt and cope. But it wasn't so easy this time: Evan is on leave and in the middle of an epic world tour with his wife Tara. I don't actually know where he is right now. He was in Bali a couple of weeks ago... If you see him say Hi!

---

As I restart the engines on All The Things, I want to thank anyone who's been waiting for an email reply, or — in the case of the 52 Things... Rock Physics authors — a book, for their patience. Sometimes it all hits at once.

Onwards and upwards!