Tuesday, December 4, 2012

Changes in location and otherwise

Changes are afoot! I'm starting as the Data Researcher at Urban Mapping in San Francisco later this month. I'm very excited to work with the team there. I'm looking forward to using the geospatial data and metadata skills I've acquired over the past six years at UC Berkeley to augment and expand UMI's data holdings and services.

Reflecting on the change, I'm especially excited to explore and use a variety of metadata schemas and standards. Library cataloging is moving in the right direction, away from digitally recreating a card catalog framework, but the changes comes slowly. I'm looking forward to the fast pace of a development environment and the new challenges I'll face.

While the focus of this blog may waver from GIS in libraries, it won't be able to stray far from libraries in GIS. I'm looking forward to sharing more frequently and using this as a work blog to keep a record of my process and side projects. I plan on staying involved in WAML and MAGIRT as I think there are some important discussions to be had across geospatial libraries and library-like places.

Friday, October 26, 2012

Catch up

What I've been up to recently:

  • Getting ready for WAML 2012 in Honolulu where I'll be serving as President of the Western Association of Map Libraries
  • Finishing up scanning over 860 maps of the San Francisco Bay Area that are in the public domain. These maps will eventually be added to UC Berkeley Libraries' digital holdings and be available for free to anyone who is interested in the maps.
  • Working with an awesome intern to georeference these maps (hoping for 285 by July 2013)
  • Working with the Data Librarian here at UC Berkeley on an automated geospatial metadata project to create metadata for GeoData@UC Berkeley, the UCB instance of Open Geoportal. We're using BaseX
  • Working with students on GIS projects and presenting geospatial thinking and software to anyone who will listen
  • Continuing to work through Learn Python the Hard Way which is very awesome: thanks Zed!

Monday, July 9, 2012

Todd Park and Frank Warmerdam



A few weeks ago, I had the pleasure of seeing both Todd Park and Frank Warmerdam talk about their work. The contexts couldn't have been more different (Commonwealth Club for Mr. Park, GeoMeetup for Mr. Warmerdam), the crowds didn't have any crossover, and the topics were completely different, but I was struck by how both of them clearly love what they do and are willing to share their knowledge and interest broadly.

Todd Park talked about health data and a bunch of other stuff I wasn't familiar enough with to report here. What was clear was that he is very excited about working in technology in the federal government and is committed to making technology work for the US. He gave generously of his time, engaging with everyone who wanted to speak to him afterward, including myself. He told each person to email him and that he read his own email and would respond within 72 hours. I've encountered less important people for whom that is a challenge.

Frank Warmerdam talked at Geomeetup at Code for America in San Francisco. I arrived just a bit late due to sampling beer at City Beer Store, but in time to get a great seat to hear Frank talk about GDAL/OGR. I just started using GDAL to transform rasters for GeoData@UC Berkeley and became quickly enamored (enamoured?) of GDAL's straightforwardness and powerful capabilities. Coming from a desktop GUI environment, GDAL is awesome.

Mr. Warmerdam talked at length about GDAL/OGR, its history, capabilities, and future development. I was impressed, again, with his willingness to spend his time talking with what amounted to some clearly knowledgeable GDAL/OGR users and a number of fanboys, of which I had just become one. At one point, he wasn't able to open a web browser in OS X and waved his hands and claimed, "I don't know how to use this, I'm not technical!" Beyond not using non-Linux operating systems, Mr. Warmerdam was also extremely giving with regard to his time and answered every question the audience asked.

I'm thankful that I live in a place where people like this, who love what they do and want to share it, are interested in coming to talk. I hope I can put my fanboydom to good use in the near future and pay forward the same infectious enthusiasm and deep knowledge someday soon.

Sunday, June 10, 2012

My first 4 days with gdal



I'm new to using the command line to get things done. Last month, I installed Lubuntu and a very small ssd in a very old MacBook because I wanted to save money and I've read a few places online the best way to learn how to code (besides spending the time to learn how to do it) is to use Linux.

This past week at work, I spent most of my time attempting to translate Esri to OGC (in Windows). I realize I'm no spatial Neil Armstrong, but nearly all of my in-depth GIS experience has been in the Esri context. This was the first GIS project I started where I had no idea where to begin (or even what the issues might be) and then, once I did start, if I would be able to successfully finish. Scary stuff!

I'm working on a project to scan, georeference, and serve over 1,100 historical maps of the San Francisco Bay Area. I need to create a workflow that interns and library student employees can learn and improve upon. Here's how I solved the problem of Esri not playing nicely with others:


  1. Georeference the scanned maps in ArcMap (the base maps can't be beat)
  2. Make sure to use WGS84 for the spatial reference; reproject raster tool for those that aren't
  3. Export the newly georeferenced map as a tiff file with world file (might need to find some more horsepower for a cubic transformation)
  4. Move the files over to gdal to create a real geotiff: gdal_translate -of GTiff -a_srs EPSG:4326 G4362_S223_1914_L3.tif G4362_S223_1914_L3gdal.tif
  5. Load the file into the Geoserver data directory, publish, then move on to create metadata
It took me faaaaar too long to realize step 2 was the key. Now that I've taken that one small step for (a) man, I'm hoping to get a real workflow together starting with a paper map and ending up with a georeferenced raster in GeoData@UC Berkeley.

Friday, June 1, 2012

Some day my data will link



On Friday, May 18, I attended the Northern California Technical Processes Group (NCTPG) annual meeting and presentation on linked data at the UC Berkeley Alumni House. I was impressed that NCTPG would host the three speakers: Walter Nelson, Phil Schreur, and Karen Coyle at Berkeley since I haven't heard anything about linked data efforts going on in the library. I'm hopeful this was the first in a series of ongoing discussions about what UCB can do to link our records.

First up was Walter Nelson who rocked the house. He spoke very candidly about how library integrated library systems (ILS) don't really do what libraries need them to do. Nelson discussed how library data is not different from other data on the web and how libraries' and library software vendors are unwilling to recognize this. In the end, he advocated for using Drupal, which was nice to hear. I did some looking around, and, of course there are RDF modules for Drupal 7. Some day, my data will link.

Second was Phil Schreur who spoke about the efforts Stanford has been making in working with European libraries and linked data. I don't remember what Schreur's official title is, but it's certainly something about metadata, a clear indication that metadata, not bibliographic data, is on the minds of Stanford librarians and administration. Some day, my data will link?

Finally, Karen Coyle showed the group some more practical ways to engage with linked data for libraries including the interesting Linking Open Data cloud diagram. Since I'm not afraid of linked data or non-MARC standards, Coyle's discussion was interesting and motivating, not intimidating. Some day, my data will link!

Overall, I'm concerned about how to move forward and link data. I'm working on a geoportal project that uses FGDC and a homebrewed schema in Solr. It's not linked data, but that's some of the only metadata I'm in charge of at the moment. My greater concern is how to persuade people that linked data is the future. It seems to be a proposition that one either inherently understands and can see the value of, or one that seems frightening, especially when compounded with the prospective time and money issues foreseen by those who control such things.

MARC is an OK output format, but it locks library data away from the way many potential library users look for data. When I find myself trying to explain to library users why they can't find material in our library by searching the internet, I have to stop short and admit it's a failing on the library's part, not the user's. It's for them that I will link my data (someday).

Thursday, April 19, 2012

WAML 13 site visit part 2


The exciting part 2 of the Tenaya Lodge familiarization weekend is here!

Saturday's schedule was full, starting with breakfast at 7:30am (there was a promise of no sales PowerPoint presentations) and ending with dinner at 7:00pm. Little did the group know we were in for a lot more . . .


First stop was the historic Wawona Hotel which was, indeed, very historic. With no TVs or phones guestrooms and huge porches engulfing every building, a stagecoach's arrival in the circle drive wouldn't be out of place here. You can see a poorly captured version of the main building above, with other members of the familiarization group making their way back to the bus.

The Wawona Hotel was followed by making our way into "The Valley," as it's called around those parts. Our group was treated to the tour guide stylings of Manny, our awesome bus driver. Manny's ability to tell stories about Yosemite, the land, the people, its history, was amazing. It would come in handy throughout the trip. In the park we stopped at Tunnel View which overlooks the entire Yosemite Valley. There are have been many amazing pictures taken from here, but none with the backs of the exact same heads as I was able to capture with my phone:


Take that, Ansel Adams!

Another interesting feature of Tunnel View is this relief model:


I'm not sure if they took wear and tear from wind and rain into account when setting the scale for the z-axis . . .

We made it to the valley floor with Manny talking about Yosemite the entire time. Stories about specific waterfalls, trees, and buildings. The bus made it to the Ahwahnee Hotel, the grand hotel in the valley. The  Ahwahnee is very nice, historic, and has a lot of stories both Manny and the hotel manager shared with us. Apparently, very late one night, there was quite a ruckus at one of the three Steinway pianos at the hotel. The manager was roused from his sleep by complaining guests. Upon further inspection, the manager found a number of guests rather drunkenly gathered around the piano, singing loudly and carrying on. When the manager approached the group to tell them to stop playing, he realized that the singer was Judy Garland accompanied by none other than Ansel Adams on piano. The manager shrugged and went back to sleep.

The  Ahwahnee is beautiful as you can kind of make out here:




After lunch, we loaded onto the bus to head closer to Yosemite Falls. I was very excited to walk to the falls since I'd come the whole way from Oakland to see some nature and had, so far, seen a lot of different hotel rooms. On the way to Yosemite Falls, we stopped at Curry Village where the famous permanent tent lodging is located. Here's a view of the exit from Curry Village:



While we were waiting for the bus to pick us up, it started to rain. Once we arrived at the main Yosemite Falls area, the rain started coming down really hard. Only a few of us were up for a wet walk to the falls. I was determined to commune with nature, even if I had to wear a rain jacket (my own) and a plastic bag poncho (supplied by Yosemite). It was a wet walk (no pictures due to the rain), but we did get to see frazil ice, which was very cool.

After recovering from the wet walk to the falls, we loaded back into the bus to see a final property which had been remodeled. The rooms were nice, but I forgot to get the name of the property as my pants were soaked (which I realize was my own fault), and I was ready to head back to Tenaya.

In the meantime, the rain had turned to snow. Manny drove/talked us back through the park toward the south entrance, the way we came in the valley. By the time we reached the road, it had been closed due to snow(!). We made another lap around the park to get another bus (our defroster wasn't working right and Manny couldn't see). We faced the prospect of a three hour drive around the mountain back to Tenaya. Which would have looked a lot like this:

When we picked up the new bus, our gracious (and VERY smart) hosts informed us that we should stock up on all the "supplies" we needed at the store for the long journey into night. That was great news! Manny got the bus together and we were ready to roll. At this point, Manny informed us that his dispatch office told him the south exit would be clear and we could make it over the pass with chains. We made another lap around the park: 


View Yosemite in a larger map

Each lap took about 20 minutes due to the snow, bus, and tourists scrambling to buy chains for their cars. Unfortunately this time, the park ranger stationed at the exit told us the road would open in 20 minutes. So we sat in a nearby parking lot, hoping this 20 minutes would be what it took to get us around the short way.

After 20 minutes, the ranger sauntered over to the bus to let us know the road would not open. We had to take the long way around. Manny, being the good sport he is, just started telling more stories, kicked the bus in gear, and we headed out for what was turning out to be very much like Gilligan's three hour tour. Unfortunately for Manny, his microphone stopped working. This did not thwart his desire to share his stories and he kept on telling them, though no one beyond the fifth row of the bus could hear anything.

We eventually made it most of the long way around, with Manny having to put the chains on and take the chains off the bus repeatedly throughout the journey. The last six miles to Tenaya tested our mettle: chains were required but wouldn't stay on the bus. Manny had to stop three times in the last six miles to adjust the chains. It was amazing.

The group rolled into the Tenaya Lodge parking lot at 9:45pm, nearly three hours late. Manny received at least three standing ovations from the group and probably had more stories to tell the next group he takes into the park. The Tenaya staff who were with us were amazing and never lost their composure. The staff at the lodge kept an amazing Basque dinner ready to go so, upon our arrival, we sat down to a delicious and much needed dinner.

The whole point of relating this story is to let you know that WAML will be well cared-for during our meeting by a professional and responsive staff at Tenaya Lodge. Even in the most difficult times during the long ride home, the staff was nothing but courteous and professional even though it was their sixth day of work in a row and they were trapped on a bus with 35 meeting planners. Also, it's a warning: don't forget to bring snow chains with you for WAML 2013!

Sunday, April 8, 2012

Pastmaper and preserving and versioning web mapping


Yesterday, I was fortunate enough to be invited to attend a meetup to discuss Pastmapper, the awesome historical web mapping project that seeks to recontextualize historic mapping with a contemporary interface. Self defined, Pastmapper "is a new platform for organizing data using the visual language of online maps to describe the world of the past. It’s also the manifestation of months that I’ve spent tracing old maps in Illustrator, learning to hack together code with the Google Maps API, and cramming 19th century city listings into a database."

The idea was to get a number of people who are interested and engaged in historical research and mapping together to talk about the project, our own work, and how Pastmapper might move forward.

One of the most engaging portions of the meetup was Eric Fischer (@enf) describing his interest in false maps of the past and maps of potential futures. These include the mapping of the mythical Laguna Dolores in San Francisco, and the plans to cover nearly the entire eastern portion of San Francisco in freeways during the late 1950s. Both of these situations ended up not being true, though mapped and published as if they were (or were going to be). 

Brian Mount (@brian_mount) discussed his work in creating a web app that allows users to modify San Francisco streets. He discussed the ability to move curb lines to make streets more narrow, close streets, and create a transportation grid of possibility rather than inevitability.

These comments and the interest generated by everyone at the Pastmapper meetup really made me think about how there is so much geospatial information generated in web mapping applications that isn't retained in any way. As today's users encounter Brian's app or planners work on creating the new bicycle utopia in San Francisco, there's literally no way to see what possibilities are created. That blindess of today's possibilities extends to tomorrow's understanding of today. If we couldn't see what the plans were for San Francisco before the Freeway Revolt of 1959, contemporary roads in the city wouldn't make as much sense. Besides, who doesn't feel smug at being able to know the future when looking at the past?

While talking about Pastmapper, which is, itself, an amazing platform for mapping the space of the past, today, false maps of the past, and possible maps of the future, I realized that there are some possibilities for GeoData@UC Berkeley and Open Geoportal to become a platform where geospatial musings can live on and be discovered in the future. It will take some work to figure out how to technically make the connections, but is something worthwhile to investigate.

Thursday, April 5, 2012

WAML13 site visit part 1


Over the weekend, I visited Tenaya Lodge and other properties operated by Delaware North at Yosemite National Park as part of the company's familiarization weekend. Since WAML's 2013 meeting will take place at Tenaya, I thought it would be a great idea to see the meeting facilities, lodging accommodations, and, most importantly, taste the food, in person before the meeting. Plus, who would pass up a trip to Yosemite?

Getting There

My family (wife Jenica and daughter Clio) and I set out on Friday morning after rush hour, planning to stop a lot along the way. After stopping in Mariposa for supplies (sandals and craft beer: Dust Bowl Brewery Hops of Wrath), we headed to Madera for another rest stop:

Another stop for lunch, and we finally made it to Tenaya!

Tenaya Lodge property tour

The first order of business was to tour the Tenaya Lodge property. We saw three levels of sleeping rooms, the indoor pool, game room, three on-site restaurants: (Jackalope's Bar and Grill (casual dining), Sierra Restaurant (with a giant patio), and Embers (fine dining), the currently-under-construction outdoor pool (heated and open all year), meeting rooms, and "the cabins," which is a separate set of buildings that house three 1-2 bedroom apartment style sleeping rooms and Timberloft Pizzeria.

Sleeping rooms


WAML has been assigned standard lodging at Tenaya. All sleeping rooms were renovated within the past year and have granite counter tops and flat screen tvs. Wifii is $9.95 for 24 hours and I had a very inconsistent cell phone signal (Verizon) in the room.There's a nice set of rocks in the parking lot where there's a really good signal. The beds were comfy with ample pillows. You can see that the chairs were suitable for reading:



Meeting rooms

Tenaya Lodge has over 15,000 square feet of meeting space. WAML is a relatively small group for Tenaya, so we'll fit in a pretty standard meeting room with air walls for the meeting, lunches, and the banquet. You can see a map of the meeting rooms on the Tenaya site as well as a capacity chart (pdf warning for both). We can bring our own A/V equipment or use theirs. It's nice to have a choice!

Food

One of the nice things about participating the familiarization weekend is that I had the opportunity to meet and talk with a number of professional meeting planners. One thing I found out from a few of them is that there is a saying about planning meetings, "As long as the food is good, nothing else really matters." If that's indeed true, then WAML will have a fantastic meeting at Tenaya Lodge.

Our familiarization weekend group (about 35 people) was able to sample a wide variety of catering services while at Tenaya. We had breakfast, lunch, snacks, heavy hors d'oeuvres, and dinner at Tenaya and other properties operated by Delaware North. Everything was very, very good. A number of chefs had recently changed properties where they were working and their interest in providing food specific to their location was apparent. You can find out more about catering and food on the Tenaya Lodge site. Suffice it to say that we'll be well fed and happy! Delaware North has emphasized well prepared, sustainable food for quite sometime. It really shows in the care they provide at Tenaya.

In addition to the restaurants mentioned above, there's the Parkside Deli which offers quick breakfasts, sandwiches, and food to go.

Look for part 2 where I'll detail the exciting adventure the familiarization group had touring other properties in Yosemite. Plus: snow, outdoor beauty, and field trip ideas!

Monday, March 26, 2012

Support for OGP!


Late last week, I found out, through inaction of my own (e.g. letting things work themselves out), it looks like library administrators and sys admins will be more fully supporting UC Berkeley's instance of Open Geoportal. This is, of course, great news as we've had GeoData@UC Berkeley up and running since the end of August, 2011.

My first order of business is to get access to at least the Solr records as there are a number of highly editable records floating around out there. I need to get these updated to remove the "REQUIRED" content in fields where no metadata was entered. Next, I'll start adding more data.

We're coming up on our first anniversary of harvesting freely available GIS data from Bay Are sources, so versioning our data can no longer be avoided. Hopefully, an annual cycle will be the longest interval we'll have for data refreshes in the future.

With more programmer time available for the next few months, I'll need to get an internship up and running for GIS. We've got 400 maps of the Bay Area that are scanned and ready for georeferencing too.

Monday, March 19, 2012

WAML 2013 site visit

I'm headed to Tenaya Lodge at Yosemite late next week (March 30) for a site visit for the 2013 WAML meeting. I'm excited to see the park as I've never made the trip and hope to find out more about the lodging, meeting facilities, and food at Tenaya.

Maybe there will be some snow?