Users navigate via a web map (Google by default but you can choose other options), and as they change the extent by panning or zooming a list of available spatial layers appears in a table of contents beside the map. Hovering over a layer in the contents reveals a bounding box that indicates its spatial extent. Several algorithms determine the ranking order of the results based on the spatial intersection of bounding boxes with the current map view. For instance, layers that are completely contained in the map view have priority over those that aren’t, and layers that have their geographic center in the view are also pushed higher in the results. Non-spatial search filters for date, data type, institution, and keywords help narrow down a search. Of course, the quality of the results is completely dependent on the underlying metadata for the layers, which is stored in the various repositories.

The project was pioneered by Tufts, Harvard, and MIT , and now about a dozen other large research universities are actively working with it, and others are starting to experiment. The purpose of the summit was to begin creating a cohesive community to manage and govern the project, and to increase and outline the possibilities for collaborating across institutions. At the back end, librarians and metadata experts are loading layers and metadata into their repositories; metadata creation is an exacting and time-consuming process, but the OGP will allow institutions to share their metadata records in the hope of avoiding duplicated effort. The OGP also allows for the export of detailed spatial metadata from FGDC and ISO to MODS and MARC, so that records for the spatial layers can be exported to other content management systems and library catalogs.

The summit gave metadata experts the opportunity to discuss best practices for metadata creation and maintenance, in the hopes of providing a consistent pool of records that can be shared; it also gave software developers the chance to lay out their road map for how they’ll function as an open source project (the OGP community could look towards the GeoNetwork opensource project, a forerunner in spatial metadata and search that’s used in Europe and by many international organizations). Series of five-minute talks called Ignite sessions gave librarians and developers the ability to share the work they were doing at their institutions, either with OGP in particular or with metadata and spatial search in general, which sparked further discussion.

The outcome of all the governance, resource sharing, and best practices discussions are available on a series of pages dedicated to the summit, on the opengeoportal.org project website. You can also experiment with the OGP via http://geodata.tufts.edu/, Tuft’s gateway to their repository. As you search for data you can identify which repository the data is coming from (Tufts, Harvard, or MIT) based on the little icon that appears beside each layer name. Public datasets (like US census layers) can be downloaded by anyone, while copyrighted sets that the schools’ purchased for their users require authentication.

OGP is a simple yet elegant open source project that operates under OGC standards and is awesome for spatial search, but the real gem here is the community of people that are forming around it. I was blown away by the level of expertise, dedication, and over all professionalism that each of the librarians, information specialists, and software developers exuded, via the discussions and particularly by the examples of the work they were doing at their institutions. Beyond just creating software, this project is poised to enhance the quality and compatibility of spatial metadata to keep our growing pile of geospatial stuff find-able.

For full details check out this flier. You can register via the College’s CAPS website; do a search for DEM and register for each session (DEM0003, DEM0004, and DEM0005).

• There will be a QGIS update, version 1.71, sometime this month; it’s a minor release that will fix a few bugs. Some future version of QGIS will included a Data Browser (think Arc Catalog).
• For folks who have asked me how they can get more cartographic production power out of QGIS, Inkscape looks like a good option – folks at UC Davis have been experimenting with it with some success.
• Learned about a documentation system for open source (or any) project called Sphinx; documents are stored as restructured text files with some Python scripts that link them together and provide formatting for output and display.
• Got a great, clear, concise overview of what’s involved with an open source web mapping stack.
• There’s a study at Idaho State (affiliated with the group of folks there that created Map Window)that’s attempted to define the core functions of GIS based on a survey of GIS users. You can view their data by contacting the project lead.
• Educators at a community college in Arizona are experimenting with an open source raster program called Opticks; a viable solution to more expensive packages like ERDAS and IDRISI.
• There are some new Python libraries you can use to create and mine KML data
• The FCC used a clever method for collapsing / aggregating US Census geography from the block level to create their Broadband Map.
• While I’ve heard of and poked around the Open Street Map Project, I never realized that many of the users were contributing to the project by walking, cycling, and driving around with GPS units, which they upload to create and update road networks around the world. They also use some free datasets (like the Census TIGER files and equivalents from other countries) to augment and provide a frame of reference for their systems.
• Data in the UK is finally opening up some more, and demand for products from the Ordnance Survey have been off the charts.
• My presentation on using QGIS in an Academic library went pretty well, and I was pleased to discover I’m not the only GIS librarian at the conference! I’ve met folks from Ontario, Alberta, and Kansas.
]]> http://gothos.info/2011/06/foss4g-in-denver-this-sept/ Mon, 20 Jun 2011 13:25:13 +0000 http://gothos.info/?p=679 I’m all set to go to FOSS4G 2011, the global conference on Free and Open Source Software for Geospatial, organized by OSGeo. The conference takes place in Denver, CO from Mon Sept 12 to Fri the 16th. The first two days (12th-13th) consist of morning and and afternoon workshops while the main conference takes place from the 14th to the 16th and features talks, presentations, tutorials, exhibits, and some fun social events.

The full program is available here, and it looks like it’s chock full of interesting presentations and lots of great learning opportunities via the workshops and tutorials. I’ll be presenting on Weds afternoon, for those interested in my adventures in introducing QGIS on a college campus.

If you’re on the fence about attending, consider this: this is the sixth year for the conference and it’s only the second time that it’s been held in North America (Canada hosted the 2nd conference in 2007) and the first time it’s being hosted in the US. So if you’re in North America and getting funding from your organization for travel is an issue, now’s your best chance to go. This is truly an international conference (was also hosted in Switzerland, South Africa, Australia, and Spain) so it probably won’t be back on these shores for awhile.

Here’s some more motivation – early registration at the discounted rate ends on June 30th!



]]> http://gothos.info/2011/05/gis-practicum-and-qgis-tutorial/ Thu, 19 May 2011 16:22:43 +0000 http://gothos.info/?p=656 I recently finished running my day-long practicum this semester, Introduction to GIS Using Open Source Software, which used QGIS to introduce new users to GIS. Each participant received a printed tutorial booklet for the class, which I’m now providing online under a Creative Commons license here:

http://www.baruch.cuny.edu/geoportal/practicum/

I plan on revising the tutorial once a year, and the online version will be one version behind what I use and distribute in class. I’ll be busy this summer tweaking the guide and the class and will offer the in-class version to members of my university again in the 2011-12 academic year.

I held the workshop three times and had 45 participants out of a possible 60. Half were graduate students and the other half were faculty or staff. The top three disciplines that were represented were public health, library and information science, and public administration; I also had a smattering of folks from across the social sciences and a few from the natural sciences. Despite my intention to introduce GIS to novices, about half of the participants had some previous experience using GIS, primarily ArcGIS. All of these folks were pleasantly surprised with how well QGIS performed.

If you have any questions or comments about the tutorial feel free to contact me:

francis DOT donnelly AT baruch DOT cuny DOT edu

There’s also the gothos email address, but to be honest I don’t check it as often as I should – frank

]]> http://gothos.info/2010/12/qgis-workshop-plans-and-updates/ Fri, 03 Dec 2010 20:52:05 +0000 http://gothos.info/?p=556 I’ve been hacking away for several months now at creating the day-long GIS practicum / workshop using QGIS that I hope to offer on my campus in the spring. I’ve finally finished it and am just working out the administrative details. My hope is that after completing the workshop, participants will have enough knowledge to then go out on their own and work on their own projects (with the tutorial manual to fall back on). The workshop will consist of five parts:
• Part 1: General introduction and overview to GIS
• Part 2: Introduction to GIS Interface (learn how to navigate the interface: adding data, layering data, symbolization, changing zoom, viewing attributes, viewing attribute table, making basic selections, difference between data formats, organizing projects and data)
• Part 3: GIS Analysis (using site selection example in NYC, basic geoprocessing tasks, attribute table joins, plotting coordinate data, buffers, basic statistics, advanced selection)
• Part 4: Thematic mapping (using US states as an example, map projections, coordinate systems, data classification, symbolization, calculated fields, labeling, map layouts)
• Part 5: Going Further with GIS (exploring and evaluating online sources for free data, exploring open source and ArcGIS software resources for learning more)

I designed the workshop around QGIS 1.5, but now that version 1.6 is out I’ll have to go back and make a few tweaks. Details about the new version and recent updates are available HERE. For my purposes, the most noteworthy changes are:

• New operators in the field calculator (like concatenate)
• Some improvements to the measurement tools
• The ability to view non-spatial attribute tables
• Support for color ramps for symbolization
• New classification schemes (including natural breaks!)
]]> http://gothos.info/2010/06/learning-python-at-pycamp/ http://gothos.info/2010/06/learning-python-at-pycamp/#comments Thu, 10 Jun 2010 19:39:09 +0000 http://gothos.info/?p=513 I got back from leave a couple week ago, and spent part of it at a Python boot camp. I’ve gotten tired of hacking away at data in spreadsheets and read in several places that Python is a good language to learn for beginning programmers – it’s also open source, flexible, and is used by many in the GIS community for processing data and building plugins and software (the instructor for the camp, Chris Calloway, pointed me to this presentation on Python scripting techniques for ArcGIS).

The workshop was a three-day event hosted at Penn State by the Triangle Zope and Python Users Group (TriZPUG). It was geared towards beginners and non-programmers (although many of my fellow classmates were IT and systems people) and provided a pretty thorough review of all of the elements of the language – now it’s up to me to tie it all together! The price was extremely reasonable (only $300 for a 3 day class!) and I’d certainly recommend it if there’s a camp in your area; although I would also recommend reading a book or taking a tutorial to familiarize yourself with the basics BEFORE attending the class; I did, and as a result I think I got more out of it than I would have had going in cold.

The next PyCamp is being held in LA in a few days, and the following one will be in Toronto from Aug 30th to Sept 3rd (although this isn’t posted on the website yet); the normal workshop is a five day affair, the one I attended was a mini 3 day version which suited my needs pretty well.

There are tons of Python tutorials on the web and Python’s site is pretty definitive. If you’re looking for a book, I’d recommend Practical Programming: An Introduction to Computer Science Using Python. Unlike the “Learn Language X” books, this one introduces you to general theory and practice in programming, and the authors illustrate the applications with practical examples using Python – it’s been immensely helpful to me. Now that I’m around the initial learning curve, I’ve been relying more on Beginning Python: From Novice to Professional, which is better as a reference book and good for illustrating many of the uses for individual objects, methods, etc (which I had a hard time grasping before I covered the basics of programming).

]]> http://gothos.info/2010/06/learning-python-at-pycamp/feed/ 1 http://gothos.info/2008/04/census-update-shapefiles-acs-estimates/ Thu, 17 Apr 2008 01:00:22 +0000 http://gothos.info/2008/04/16/census-update-shapefiles-acs-estimates/ I’m in Boston at the Association of American Geographers (AAG) annual conference this week, and attended a great series that explored what the Census Bureau is currently up to. Here are some hi-lites:

The Bureau is now providing the TIGER line files in shapefile format! Before, it was only possible to get generalized cartographic boundary files directly from the bureau in shapefile format. Now, you can get the boundaries in their original detail from a public domain source. Includes 2000 census geography plus some updates for 2007 for states, counties, metros, places, zips, districts, pumas, and more. Currently, it does not include tracts, block groups, or blocks.



The 2008 release of the American Community Survey (ACS) will include two datasets. There will be the annual numbers for geographies that have over 65,000 people, and for the first time there will be three year averages for geographies that have over 20,000 people. In each succeeding year, this average will be recalculated by adding in the most recent year and dropping the oldest one. Data for geographies with less than 20,000 people will become available in 2011 and will be based on five year averages. The good news is that, from that point forward, data will be available for all areas every single year. The bad news is that the long form (the one in six sample of households taken in the decennial census) is being discontinued and will not be conducted in 2010. Census 2010 will consist solely of the short form questions (the 100% count that covers the basic demographic variables). The ACS will serve as the replacement to the long form, but in most cases the data will not be suitable for making historical comparisons (i.e. comparing 2010 to 2000).

Bureau reps gave an overview of their Population Estimates program. Unlike the ACS which is survey based, estimates are calculated using a cohort component analysis that accounts for births, deaths, and migration each year. Estimates are calculated nationally and at the county level. The county numbers are used to create estimates for each state, which are then adjusted to fit national numbers. Data is available for total population, race, age (broken down by gender for each year at the national level and for five year groups below that) and housing units. Some data is also available for metropolitan areas (which are county based) and county subdivisions (for total population only).

The Bureau gave an overview of Dataferret, which is a tool for data power users. It is available in two versions, as download-able software or as a browser-based JAVA applet, and allows users to gather and process data from several different government sources (unlike the Amerivan Factfinder, which focuses solely on downloading census data).

Finally, things are ramping up for the 2010 Decennial Census. The bureau is updating its master address files and has almost finished recalibrating the TIGER files for each county, so that boundaries are precise within a maximum limit of 70 meters.

]]> http://gothos.info/2008/04/hands-on-gis-census-workshop/ Thu, 10 Apr 2008 18:27:25 +0000 http://gothos.info/2008/04/10/hands-on-gis-census-workshop/ I’ve posted the tutorials from the workshop I gave the other day for the NYCRDC. I’ve created a Resources page to hold resources hosted on this site – you can find them there, along with the datasets.

Overall I think it went rather well, but it was way too much material for a three hour workshop! We covered the intro slides, and Part I (Intro to GIS and ArcMap). I did an abridged version of Parts II (Intro to Layout View) and III (finding and downloading data, ArcCatalog, preprocessing in Excel) rather than doing all of II and none of III. The third part covers a lot that the standard ArcGIS texts gloss over (or leave out all together), so I really wanted to cover some of that material. But I couldn’t omit any of the basics in the first two parts, because you really need to know them before you can delve further (and understand why you’re delving). Ahhh, the steep learning curve of GIS!

]]> http://gothos.info/2008/03/nycrdc-2nd-annual-workshop/ Wed, 26 Mar 2008 01:39:28 +0000 http://gothos.info/2008/03/25/nycrdc-2nd-annual-workshop/ I attended the first of the three workshops held at Baruch College as part of the New York Census Research Data Center’s 2nd Annual Workshop series. The NYCRDC provides confidential census microdata to researchers at secure facilities at Baruch and Cornell.

This year’s theme is census geography and mapping, and there were a number of great presentations that covered census geography from the global down to the block level. My personal favorite was a presentation that illustrated the composition and evolution of census tracts – using Legos! Not the real ones mind you, but digital photos of Legos that were enhanced and tied together with Flash in a Powerpoint presentation.

I have provided a link to the 2nd Annual Workshop page before – but there it is again. Powerpoints, and perhaps video footage, of the presentations should be posted there relatively soon.

I also gave a promo to the hands-on GIS workshop that I’ll be doing as part of the second workshop of the series. Two weeks to go, and I still have a lot to do…

]]>