Last week the City and County of San Francisco hosted a Housing Data Jam. Led by Presidential Innovation Fellow Ian Kalin, who just finished a tour with the White House Office of Science and Technology Policy, the half-day event was co-hosted by the San Francisco Mayor’s Office of Civic Innovation, the California Governor’s Office of Business and Economic Development, and Code for America. Held in City Hall, the event brought together leaders with many different perspectives on the housing issues challenging San Francisco, from nonprofit leaders to energy efficiency experts.

The Data Jam was inspired by Mayor Edwin Lee’s 2013 State of the City address and the recent passage of a new Open Data Ordinance by the San Francisco Board of Supervisors. Based upon the principle that “open government data has the power to create real economic value,” the invite-only event gathered investors, policy experts, executives, technologists, and entrepreneurs together to brainstorm new applications that could be developed using open data.

“Open data has the power to fuel innovative new products and services that can help people while also creating new jobs,” said Kalin. “Events like these Data Jams are helping to put public data resources in the hands of people that have the skills and civic motivation to rapidly transform the data into valuable tools.”

Getting Down to Work

The Data Jam began with opening presentations from a number of federal, city, and state leaders, including Kish Rajan, Director of the California Governor’s Office of Business and Economic Development, who described open data as a “playground for brilliant people.” Attendees were then split into five groups and given general topic areas to innovate around, including homelessness, building safety, health code violations, energy efficiency, and new construction development.

I worked in the “Building Safety” group, alongside tech entrepreneurs, civic activists, city leaders, and an Obama Administration Open Data Initiative leader. Everyone contributed ideas and information. In the end, we came up with a mobile app called “Safe Block” that aggregates a variety of publically-available safety information about a property, such as soil and water toxicity ratings, crime activity, traffic incidents, and more. A person looking to rent or buy a property could enter an address and quickly understand how healthy or safe their environment might be. 

After meeting for a couple of hours, each small group then reported back to the larger group and that is where the magic happened. The incredible wealth of knowledge and passion in the room quickly became clear. For example, nonprofit leaders shared the realities of putting Section 8 Housing vouchers to use when little housing is available. City leaders suggested existing resources that could make projects flow more smoothly. And, entrepreneurs pointed out the business possibilities certain ideas opened up.

Once the group had prioritized and consolidated the projects, participants volunteered to give their time and talents to making them a reality. According to the Data Jam format, every idea will have 100 days to develop into a working prototype and earn the chance to be featured by the event coordinators and the Obama Administration’s Open Data Initiative at a celebratory “Datapalooza” later this year.

“Open data can highlight the often invisible factors that contribute to people’s housing experience, such as safety, health, and energy use,” said Jay Nath, the City of San Francisco’s Chief Innovation Officer. “And, open data can be a way for entrepreneurs to leverage freely available resources to develop products and solutions around housing.”

A Springboard for Further Innovation

The energy at this event was moving. Everyone there wanted to improve San Francisco citizens’ housing options and everyone had something unique to contribute to that effort. And, for many, thinking about the problems in the context of open data was a first but they embraced the idea quickly.

“Even with the excitement at the Data Jam, the best is yet to come,” said Kalin. “The Datapalooza that will follow in less than 100 days will showcase the civic entrepreneurs and also feature newly available open data sets and new challenges from the sponsors.”

If you would like to see all of the ideas created by the group, plus more, go to the ImproveSF website. You can also look for updates on Twitter at the hashtag #sfdatajam.

Seattle, Baltimore, Atlanta, DC, San Francisco, and Chicago – join Socrata as we share what we’re seeing on the National Day of Civic Hacking.

Want to send us a quote or pic from your event? We’re happy to include it here. Email us at blog@socrata.com.

Need technical support? Write us at support@socrata.com. Want to see more photos? Look here.

Live Blog

Steve Ellsworth @2 pm ET (Sunday)

From the Atlanta National Day of Civic Hacking, I have the following P’s to share:

People: There are a lot of people with unbelievable talent who were involved this past weekend, and more who want to in the future.

Passion: These people are driven to do good for the benefit of humankind; it’s not about them. 

Proficient: What many teams created in the span of 24 – 36hrs was amazing. The Atlanta project that won is a great testament to what can be accomplished

Perseverance: No matter how daunting the challenges are, they just figured out how to get solutions created (albeit partly due to an abundance of Red Bull and Mountain Dew). 

Bridget Quigg @ 2:00 pm PT (Sunday)

San Francisco Day of Civic Hacking, hosted by San Francisco State University, is full of great ideas to help developing countries in agriculture and healthcare. So far we’ve heard responses to two challenges from the Peace Corps. One group just presented a way to help the very rural coffee farmers know the best price they can get for their crop via SMS. And, another group wants to make it easier for both nurses and patients know the locations of medical supplies in their country using text messages.

This event was a great example of how successful growing the open data movement means passionate people need to step up and spread the word. In this case, I want to give a big kudos to organizer Julio Feliciano, as well as Sameer Verma and Pheobe Kwon, for committing themselves to making something happen. Congrats!

The event was held at the J. Paul Leonard Library on SF State’s campus.

App presented that chronicles ballot initiatives in SF and follows up on follows up on how those that are passed get put into action, e.g. time frame, costs, and more.

MedLink uses text messaging capabilities on phones to communicate up-to-date medical supply locations in Gambia and other developing nations.

Coffee farmers have the opportunity to negotiate a more fair price for their crop by receiving text messages about prices for the season.

Beth Blauer @ 4:00 pm ET (Sunday)

The State of Maryland boasted three events for the National Day of Civic Hacking. These included the AnnapolisWiki Editathon, Hack for Change Baltimore, and Hack for Change Montgomery County.  Socrata’s Beth Blauer attended the event in Baltimore and served as a judge. To quote her on twitter, “Watch how judgy I can be.”

Baltimore Mayor Rawlings-Blake welcomes the attendees of the event.

Blauer had these notes about the winner teams at the event:

The first prize awarded was the Most Transparent App. The Sunlight Foundation sponsored the $250 dollar prize for the application that does the best job of liberating data or otherwise improving government transparency. This award went to one of Baltimore’s veteran hackers, Shea Frederick, for an update to his site http://baltimorevacants.org/ that pulled data in from the Baltimore Neighborhood Indicators Alliance to show detailed demographics in Baltimore neighbors against the landscape of vacant properties and lots. 

The second prize awarded was for the Best Philly Fork for the individual or team who best duplicates a civic app already in use by the fine people of Philadelphia. Our event was done in conjunction with the City of Brotherly Love. This http://bmoremapped.com.s3-website-us-east-1.amazonaws.com/ is the beginnings of a property tax map that frees opaque city property tax data in a way that allows citizens to see if they are paying taxes that are comprable to their neigbors. It was forked from this highly regarded Philly map http://apps.axisphilly.org/avi-map/. 

The third and final prize was for the app or map or idea that could do the most good. There were two finalist that were so good we had to call a tie and in the pinch gb.tc upped the anty so that we could give a generous payout to both teams.

The first winner was Code for Sex. For those of you who don’t know, Baltimore has a serious sexually transmitted infection problem. We are the syphilis capital of the world. Code for Sex is an application that will allow residents to get tested for STIs at their doctor’s office and be notified of their results online instead of having to come in for a follow up appointment. Currently, only 40 percent of patients come back to learn if they’re infected. In addition, the application allows you to anonymously notify sexual partners of your results. It’s being backed by the health department and was really well thought out. 

The second winner in this category was for an application that was pitched by a librarian in the city’s public library. She wanted an easy-to-use application to help her users craft a resume. In her own words, “LinkedIn is too technical and confusing for the regular clients in her Job Center at the library.” As a result, a team of hackers joined her and created a beautiful and simple guided application that allows for users to answer very basically worded questions and outputs an entire resume in a Word document. It is awesome and I can see it being useful in employment one-stops and libraries all over. 

It was a long and crazy weekend but it was equally inspiring and motivating. I got to close the program tonight and I reiterated my love for this, the greatest city in America. It is so amazing to see in a city that has such a diverse set of problems that we can unite to create a diverse set of solutions. What a great life.”

Vacant lots app updated at the event by Shea Frederick.

@11:47 AM PT | TowText: beautifully simple app from Tony Webster @webster

Get a free text message if your car is towed or relocated by the City of Chicago. Check it out: http://towtext.com/

Marcus Louie @ 1:30 pm ET (Sunday)

Hack for Change DC gathered about 80 people at Google to hack and about another 50 or so at Affinity Labs nearby. 

Group that focused on a app that allows people to apply for Section 8 housing in many parts of the DC through a single form. 

This group worked on the question:  How do you create a common housing application that can be used to apply for any one (or all) of the 107 Section 8 housing buildings in DC?  Next steps are planned with Bread for the City, the non-profit who proposed the challenge next week.  When the solution is deployed, it can start collecting information on housing applications.

(Marcus Louie from Socrata is third from the left in the back row.)

Saf Rabah @ 1:20 pm ET (Sunday)

NBC covers the #hackforchange event in Kansas City (https://data.kcmo.org/)

Clint Tseng @ 12:55pm ET (Sunday)

Check out the #hackforchange final project results for the youth-focused @adlerskywatch NDoCH event in Chicago: civichack.adlerplanetarium.org/results/

Clint Tseng @ 12:30pm ET (Sunday)

Final presentations at Alder Planetarium in Chicago for National Civic Day of Hacking.

Bridget Quigg @ 4:40pm PT

Feeding the hungry hackers in Oakland.

Bridget Quigg @ 4:10pm PT

At Open Oakland’s event, more than 70 citizens attended, plus city staff. They came up with 220 questions as a group and found online answers more than 160. Nicole Neditch from the City of Oakland had a highlight moment when an attendee told her that getting involved in the challenge made him feel inspired, and also more aware of how complex government issues are. Shiela (pictured below) is a CfA Fellow in Oakland and helped to host the event.

Bridget Quigg @ 3:50pm PT

At Open Oakland’s event at HubOakland, attendees have brainstormed typical questions that citizens need answered. For example, “Where is the closest park to my house?” They’re brainstorming questions and searching for their their answers online. All of this information will feed a web application called “Oakland Answers.”

Saf Rabah @ 2:44pm PT

A few more visualizations for the FEMA Fire Data Challenge: Map of top 100 ZIP codes for Total Fires and
Chart of #Fires by ZIP code (filtered for 98xxx in WA)

Saf Rabah @ 1:32pm PT

First App of the Day! Federal Emergency Management Agency – (@FEMA) posted a challenge on hackforchange.org asking developers to build an online data visualization tool that allows the public and fire departments to view fire incident data summarized by individual and/or multiple zip code. The folks at Socrata built this cool app using DataSlate.

Way to go Darrel Cabales!

Saf Rabah @ 12:49pm PT

#hackforchange Seattle getting started at beautiful city hall. Thanks @Seattle_City for supporting #opendata developers.

Bridget Quigg @ 12:26pm PT

National Day of Civic Hacking banner at the Atlanta event.

Bridget Quigg @ 12:11am PT

Roll up your sleeves and code

* Message from a team member at the CfA offices in San Francisco.

Bridget Quigg @ 11:52am PT

San Francisco State will be hosting a two-day hackathon over the weekend. They have received hacking challenges in from the ACLU, Peace Corps, the Census Bureau, and even more organizations, and they plan to mine data from the City of San Francisco’s open data portal. Here are the organizers (left to right): Sameer Verma, Professor of Information Systems, Julio Feliciano, Web Developer, Phoebe Kwon, Chief Technology Officer.

Bridget Quigg @ 11:32am PT

Code for America Chief of Staff Abhi Nemani and City of San Francisco Deputy Innovation Officer Shannon Spanhake spoke to the group at the National Day of Civic Hacking Kickoff Reception in San Francisco. They highlighted that this weekend’s events will involve more people simultanteously civic hacking across America than ever before. Spanhake said, “For those of us who work in government, this feels like our own national holiday”.

Bridget Quigg @ 10:21am PT

Leaders from Code for America, the Peace Corps, the Census Bureau, San Francisco State and other civic organizations gathered at the Code for America offices in San Francisco on Friday night for a kickoff event for the National Day of Civic Hacking.

Saf Rabah @ 9:04am PT

From @chrismetcalf, Socrata’s developer evangelist: Bookmark our developer API tips on hackpad

Includes links to client code libraries on github, datasets on communities.socrata.com, and developer documentation on dev.socrata.com

Follow us on Twitter: @socrata and @chrismetcalf Happy coding!”

The National Day of Civic Hacking begins in less that 24-hours. What better place to get a feel for what’s happening and what’s possible than the Code for America (CfA) offices in San Francisco?

Today, we spoke with Hannah Young, Program Coordinator of the National CfA Brigade, about what’s going on in the Bay Area and her hopes for the events all over the country.

How is the weekend shaping up in terms of events?

The brigade, our network, is organizing 34 of the 96 events — which means that a third of the events have really strong CfA connections; either brigade members, fellows, or organizers who have reached out to us to put on a CfA event. And all of the Brigade events will have representatives from the local governments in attendance.

You can view a list of the Brigade events here – and all of the NDoCH events here.

Can you tell us about the event you are attending this weekend?

I will be at the event in Oakland. It’s run by one of our former fellows, Eddie Teja, Steve Spiker, who’s a really good friend of CfA, as well the Nicole Neditch from the City and other city staff. And, our fellows are also helping us organize it because we have fellows in Oakland this year.

Tell us about fellows?

A fellowship is an eleven-month program where technologists take a year off to code for America. It was the program that started CfA.

What’s happening in Oakland?

They are re-depolying Honolulu Answers (answers.honlulu.gov). This was the app that one of our 2012 fellowships teams created for Honolulu. It’s an intuitive search for government websites. For example, if you need to renew your driver’s license, instead of going on the government site and searching for “driver’s license,” it’s a question-based search. So you would type, “How do I renew my driver’s license?”

It’s a neat story, too, because it was built during a fellowship in Honolulu and now we’re deploying it through the brigade to a different city. That’s what we’re hoping to see with all technology – not re-creating something when it already exists. That’s the beauty of open source – and why everything created at Code for America is open source.

How will people participate in Oakland?

It will be an un-conference up front, with lots of ideation and talking. We’ll be coming up with the answers that aren’t even online, yet. There is a lot of domain expertise with city folks that isn’t on the website, yet. This is just the kickoff for an ongoing push. It’s a significant app.

What are your thoughts on the events across the country this weekend?

I like to describe this weekend as a galvanizing moment in this ongoing movement around civic technology. This is an amazing display of the power of civic hacking and what we can all do together. What is also great to watch is how this momentum is going to continue and build.

Ongoing movement is the way to describe it because there has been a ton of work that people have been doing before this that we want to credit. There will also be a tremendous amount of work coming out of it.

Why is civic hacking important?

There’s all of this creativity and we’ve had this boom of ability to influence and share our opinions and engage with each other in the private sector. When you think about what that could mean if that happened in the public sector it’s really exciting. We like the idea of bringing technology to our cities and helping facilitate using it well.

You can view the history of civic hacking here – we encourage everyone to add to it.

What can you tell us about the event tonight?

We wanted to have a kick off for our Bay Area events because there are so many. We’re hoping to get that positive energy going heading into the weekend. I think that everyone is really geared up but when you showcase how connected people are it’s really exciting.

Any last thoughts on the weekend?

I’m super jazzed to see what comes out of this weekend. I hope that this energizes people about civic hacking in the same way that we feel energized here at CfA.

If you want to see what’s happening at NDoCH Brigade events throughout the country this weekend take a look at our Tumblr.

An open data project is a large and exciting undertaking. One that can truly change the relationship between your agency and the public. It can also improve the efficiency of your internal operations. But to do it well, it’s essential to gather a team with the right skills to ensure success, from strategic planning to community outreach, not to mention technical expertise.

If your agency is just getting started, a small and enthusiastic team may be all you need to launch a successful pilot. With the right mix, this group can also provide the energy, leadership, and strategic planning to get your entire organization onboard for an ongoing effort toward data sharing and transparency.

Gathering this team is not always easy. Part of the process will likely require you to sell your program to the stakeholders who may not fully understand the value of open data yet.

Fortunately, there are many clear and direct benefits for each of the open data stakeholders on your team. The team building phase is your opportunity to teach and inspire them to think about these benefits and the many exciting ways open data can improve your agency’s work. The time you spend winning them over will pay for itself, as each stakeholder can then inspire their own teams for broad support down the road.

When putting together your open data team, keep the following five groups in mind – and what’s in it for them – to ensure the skills and input you need are at the table.

1. Chief Executives
City or County Managers, Chief Operating Officer, Head of Planning

What’s in it for them: Launching an open data program is an opportunity for your organization’s executive to make a big difference in improving transparency, citizen services, and overall performance in your organization.

Their role: The executive’s primary role in the initiative is to secure funding for the program and make sure it delivers on its goals. Once they’re onboard, they can also provide the organizational leadership and impetus for your entire agency to get involved. Read more advice about how to get your chief executive’s support.  

2. Internal IT and Data Teams
Chief Information/Technology/Data Officer, IT Leaders, Developers, Webmasters

What’s in it for them: This group will be excited by the opportunity to improve efficiencies and modernize information-sharing infrastructure. This will help them streamline their everyday work. It also allows them to produce tangible results in the form of apps and interactive visualizations that help improve life both at their workplace and in the greater community.

Their role: This part of your team provides the technical know-how needed to create your data publishing plan. The chief information officers along with other IT leaders should take charge of selecting and implementing the open data technology platform. These leaders, as well as others in your IT department, may also help rally the support of business users and program leaders to deliver their information and services through the open data portal. They will also be key players in reaching out to the private developer community for input.

3. Communications and Information Officers
Public Information Officers, Communication and Marketing Managers, Public Affairs

What’s in it for them: Communications teams will be enthusiastic about the opportunity to support and share the agency’s mission with easy-to-use information, and to use their skills to help promote community engagement.

Their role: This is the branch of your team that will help you create clear, concise, and targeted messages for the many different groups who can help with (and benefit from) an open data program. Their key roles include leading external communication, acting as the press contact, and supporting community evangelism. Learn more about how this team can help engage the community. 

4. Management and Team Leaders
Department Heads, Agency-Level Program Managers, City and County Clerks

What’s in it for them: This group will be able to benefit directly when open data sets allow their teams to do their work more efficiently. When published creatively, in the form of interactive charts or visualizations, team leaders and department heads will be able to see more clearly the effectiveness of their teams and point to data-driven results.

Their role: This group provides access points for each team at your organization. By engaging them, you will be able to more effectively boost each team’s ability to deliver data and information that supports programs, your mission, and improve organizational efficiency.

This group may be the smallest at the beginning of your initiative, or during a pilot. But as your initiative grows, expanding the amount of input from department heads and creating channels to listen to all levels of your organization will become increasingly important. Learn more about how and why it’s important to engage every department.  

5. Developers in the Community
Open Data/App Developers, Open Data and Open Government Advocates, GIS Professionals

What’s in it for them: Publishing data gives developers and advocates in the community the resources they need to do their own creative work. They will take the data you provide to create tools that improve life of their community.

Their role: From providing feedback about what data you should publish to taking that data and creating user-friendly applications, this group can be an invaluable asset to your open data program. They provide a broad base of expertise and professional perspectives that your organization alone, no matter how large, couldn’t provide. To help you get this group onboard, we’ve identified four essential things you can do: publish data, connect with civic developer organizations, host a hackathon, and be humble. Read more about the four essentials of developer evangelism. 

You’ve decided now is the time for your agency to get started with an open data program. But how do you decide which data sets to publish first?  

There is no standard formula that applies to all organizations. Depending on your program goals and priorities, there can be any number of good places to start. However, in working with a variety of organizations, we have identified a few smart starting points that have worked well. We hope they will help you decide the best route to take as you kick off your open data initiative.

1. Publish data sets that help fulfill your organization’s mission and goals

Drawing clear connections between your data program and your organization’s mission will help win support, as well as give your program purpose, discipline, and measurability.

Start by looking over your organization’s strategic plan or the governor’s action plan. Then, identify within it potential data sets that would have the biggest impact on achieving those goals. You may also want to ask your co-workers what data will help them to do their job better or more easily.

As an example, the Oregon Marine Board used open data to help achieve their strategic goal to educate citizens about boating safety, education, and access. They used their open data platform to create interactive maps that let citizens know information such as location-specific waterway regulations, where to register a boat, where to purchase various permits, view boat ramp locations, and more. This ended up saving the Marine Board time, materials costs, and labor costs – all while using data to achieve a mission-based goal. Read more about the Oregon Marine Board’s open data experience.  

2. Publish data that is already (or easily) available and organized

Open data may be new, but publishing data online is not. It’s more than likely your agency already has some data available, often via PDFs, Excel spreadsheets, or other offline database extracts, shapefiles, and KML files. Since they’re already organized and perhaps online already, these are great places to start for your open data project.

Starting here means that you’ll begin by republishing these sets in all the required formats, so that they are easily accessible and machine-readable. (Learn more about data formats and standards.)  

Another way to approach this starting point, especially if you’re currently publishing very little data online, is to start with data that is easily made available. Usually, this will be financial transparency and personnel data about your organization. That’s because it is typically well-organized and can be easily extracted from your back-end financial systems as clean spreadsheets. Once it’s loaded on your platform, you can use it to build helpful visualizations and guided browsing filters to help citizens interact with and understand the data with little effort.

One good example of this is the Ethics section of Data.gov. There, citizens can search, download and visualize extensive campaign funding and other data. Before publication on the open data portal, this would have been available only in spreadsheets or by submitting a public information request. Now, the data can be easily filtered and viewed through a variety of customizable charts, tables, and sorting features, delivering on the promise of transparency and easy access for citizens.

3. Publish the information citizens and developers are asking for

This is one of the best places to start if your agency wants to increase citizen satisfaction and reduce repetitive handling of frequent information requests. There are many ways to find out what information citizens are searching for or requesting most, such as:

• Analyze your website data, using programs such as Google Analytics
• Review your Freedom of Information Act requests
• Survey your internal public information officers, clerks, and disclosure teams
• Directly request feedback from constituents and developers

Using feedback as your starting point is an organic process. After gathering suggestions, it will require prioritization, often based on strategic goals or how much work it will take to provide the requested data set on your portal.

What is exciting about starting here is that it kicks off your initiative with widespread engagement. It will also help you measure the interest level of your community on a variety of community concerns. These may surprise you! From recycling data to health inspection data, take a look at what other open data leaders have learned about which data sets the public finds most useful.  

“Energy City” Edmonton, Canada just produced another great example of government performance management through open data. Using Socrata’s GovStat program, the City is complementing its “The Way We Move” Citizen Dashboard, Edmonton now offers a dashboard for “The Way We Live.” 

A History of Success and Innovation

On July 7, 2010, Edmonton’s City Council approved The Way We Live: Edmonton’s People Plan, a 10-year strategic initiative that helps clarify the role the City has in quality of life issues. The plan is the third addition to The Way Ahead: City of Edmonton’s Strategic Plan, a collection of 10-year goals to improve Edmonton’s livability by creating a socially sustainable, civil, and caring society. 

The Way We Live Implementation Plan includes six goals. They promote improvements in connectedness, leisure, affordability, safety, attractiveness, and sustainability in the City of Edmonton. As the City moves forward to achieve these goals included in “The Way We Live,” it hopes to create a community where the residents have opportunities to thrive in a diverse and inclusive society.

Open Data’s Starring Role

To communicate about its efforts on “The Way We Live,” the City has created another online “Citizen Dashboard” (the other one is for “The Way We Move“) that shows performance results on services related to “The Way We Live.” Citizens get easy access to relevant and up-to-date information on important services such as fire first unit on-scene response time, construction waste recycled, flood proofing and odor control, and the percentage of pets that have been rescued and returned to their owners.

Fueled by open data and created in Socrata GovStat, the Citizen Dashboard provides an outstanding level of transparency into the government’s performance. Everyone can see the performance targets, whether or not they’ve met those targets, and which municipal services still need improvement. There is no filtering of the results. The data is fed automatically into the Citizen Dashboard, offering raw, accurate, real-time information. For example, the Fire Rescue’s goal for Total Response Time is to be under 7 minutes, 90% of the time. The Citizen Dashboard shows us that they’re currently at 82% in the first quarter of 2013, an improvement from the 74% in Q1 2010.

The City also provides a series of videos designed to teach the basics of interacting with the data present in the Citizen Dashboard. Citizens can comment on the datasets, use them to create charts to share or embed, and even find a specific piece of information in data sets by using the search function. The data available in the Citizen Dashboard can help Edmonton’s residents understand the municipal services provided to them, as well as give them specific talking points when they’re reaching out to their local government representatives with any concerns or suggestions they may have.

———————————————————-

Like Socrata on Facebook.

Follow Socrata on Twitter.

Data.medicare.gov is the Center for Medicare and Medicaid Services’ (CMS) flagship portal for sharing valuable data about health and healthcare in the US. The site is a key component of CMS’s efforts to align with the Digital Government Strategy and make valuable data open, accessible, and machine-readable by third parties, and as part of its broader open data efforts to improve health and health care.

Socrata worked with CMS to redesign data.medicare.gov to better serve both the needs of the site’s core data customers, as well as more casual users. The data needed to be available and accessible. The new design allows researchers, the private sector, policymakers, and others to easily browse, view, and download useful data on a broad range of topics related to health and health care

We spoke with Joe Pringle, a client services director at Socrata, about the redesign.

What are the goals for the site overall?

CMS uses this site to publish tremendously valuable data sets on hospitals, nursing homes, home health providers, and other topics. CMS data is used by a wide variety of researchers, policymakers, and the private sector to understand issues of cost, performance, access to care, quality, and other aspects of our health system.

Given the tremendous amount of change we are going through right now with health care reform, these datasets are tremendously valuable as we try to understand how we can improve health care and health outcomes. In open data we sometimes see cases where data publishers share data just because it exists, with little or no societal impact. That’s not the case here. CMS data is having a huge impact on an extremely important issue.

What were some of the goals for the redesign of the site?

CMS considered user feedback to better understand who uses the site, how they use the data, and how to make the site even more useful to them. CMS found that core data user needs were very different from those of more “casual” data users.

Researchers who really want to take CMS data and use it in their own analytical tools like SAS or R want to quickly download entire datasets in certain formats. Having the schema for these data sets to stay consistent and predictable makes it much easier for researcher to work with data that is released on a periodic basis without having to go through a complex import process each time. In addition, many of the serious data users were specialists, and just needed one dataset on a particular topic. We needed to define better pathways for these users to get quickly to the dataset that they need.

At the same time, the site also needs to serve more casual users who are coming to the site for quick facts, charts, or aggregated datasets, and are more likely to need to browse both across and within categories to find what they want. They also need to be able to preview data to understand what’s there (unlike hard core researchers and data scientists who tend to know the data extremely well without having to “see it”).

Based on these findings, Socrata worked with CMS to redesign the site in a way that greatly improved the experience of all users, providing direct access to full datasets that the serious data customers want, as well as enhancing the experience for users less familiar with the data or less experienced in working with data. We like to think of it as “democratizing” access to data and making sure that a data publisher is reaching the broadest possible audience with their data. This audience can, in turn, discover, access, use, and republish it in many different ways.

We redesigned the homepage to quickly allow users to understand what data is available, and then give a more tailored experience within each category of data to make sure we’re serving the wide range of users we care about. So far we’ve gotten great feedback from visitors to the new site.

What’s next for CMS and open health data more broadly?

This site is a key part of CMS’s overall effort to align with the digital government strategy, and, more importantly, to have the biggest possible positive impact they can on the public’s collective understanding of health and health care. We are at the beginning of what will likely be a revolution in the availability of health-related data, and CMS will undoubtedly continue to play a key role in publishing their very valuable data for everyone’s benefit.

We also expect an explosion of innovation around open health data that is powered by APIs. CMS is already out in front on this by enabling every dataset they publish with APIs that third party developers can use to connect directly to the data to build health-related applications and tools. As more and more open health data comes online, we will likely see entire industries created, much like NOAA has helped create a thriving industry around open climate and weather data.

———————————————————-

Want to read more stories of open data innovation?

Like Socrata on Facebook.

Follow Socrata on Twitter.

Looking to hack this weekend?

We were lucky to grab a few minutes to interview Tim Dupuis, Interim Director of the IT Department and the Registrar of Voters in Alameda County, about the hackathon coming up this weekend on April 27th.

The event will be located on the Berkeley High School campus, right near the University of California at Berkeley. Alameda County has a nice collection of sponsors gathered up and has high hopes for an even bigger, more productive hackathon than its first.

Socrata is sending down Product Lead Clint Tseng to be on site. When asked about his plans for Saturday, Clint said, “I’m always excited to see all the new ways open data developers want to leverage data and technology to do amazing things. It’s a great way to catch up with the community and get feedback and fresh ideas directly back to the team.”

The following is a transcript of our interview with Dupuis. See you on Saturday!

Socrata will be on site. What are your thoughts on that?

Socrata on site is really beneficial because having somebody like Clint is going to bring a technical element to it to help promote all the different tools that are on the portal, available through Socrata. It will help people integrate that data into their app directly.

In the first apps challenge, we were finding that people were wanting to download all the data off the portal into their system to build their app. By having technical support from Chris Metcalf from Socrata we were able to explain to them, “There are APIs. There are different ways of handling the portal in order to build your apps so you can get that live data in real time rather than actually having to pull it down.”

How did the last hackathon go?

The last one exceeded our expectations wildly. We had planned for about 40-50 people to be there but had, fortunately, booked a big venue hoping it would be much larger than that. And, it ended up being 120 people that showed up.

The last one we were planning to have six apps be developed and presented and we ended up with 24. And, as it turns out, we kept in contact with people who were runners up and winners. We have at least six working apps as a byproduct of that first hackathon.

What are you looking forward to at this one?

This next one, we’re not expecting anything less. In fact, we’re shooting higher than that.

It’s a different venue. One of the things we wanted to do was to move the apps challenge around the county to promote different parts of our community. This time we’re having it at the Berkeley High School. It’s a brand new building within the high school. We’ve been working with the tech staff at the high school. They’ve assured us there is going to be plenty of wi-fi bandwith.

It’s a great setting to create something for the community at large. But, certainly, it draws some interest also from the University of California at Berkeley so we’re hoping to see some of the university students coming into play.

We’ve also got the high school itself. They’re excited about it and hoping to send in some teams.

What apps do you hope to see at this event?

In the last event, our winning app was called Bookit, which was appropriate because we were in the Castro Valley Library. When you’re shopping for books, the app would let you scan the barcode on the book to see if it was actually in the library. Then, it would use your library card to check out the book for you and give you driving directions to pick it up.

The other one was done by some high school students. It would help you find parks that meet specific needs like, if you want to walk a dog, this is a dog friendly park. Or, do you want to have a barbeque? If so, where do you want to have that? That app, again, would give you driving directions on how to get to the specific park that you are interested in.

We had another one that helped people who were on food stamps find places where they could use their food stamps to find healthy food. Because, that’s a challenge for that community. So, we had some really nice apps from that first event.

At the new one, since we’re having it in Berkeley, you might see something around disaster preparedness. The city itself, on that same day, is doing an earthquake preparedness event so we might see some stuff around safety.

Since it’s at a high school, we might see something around education. Also, being close to Cal Berkeley, we might see something that is of use to a university setting.

Will you be on site?

I will be. I am co-MCing the event with Alameda County CTO Tobin Broadhurst. Keith Carson, Supervisor District 5 will be a keynote speaker. Susan S. Muranishi, the County Administrator and Mayor Tom Bates, City of Berkeley will also be in attendance and will welcome attendees at the event.

Who would you like to encourage to join the event?

We are encouraging developers and also journalists and people with ideas. We like to say as we’re promoting it that you don’t have to be a developer to be at the event. The ideas are just as important as the skills to build the ideas.

What I like to say is that the developers that are coming to these events really enjoy coding but that doesn’t mean that they have that killer app idea. They are looking to participate with someone who is a business driver, a visionary in a certain area, so they can code to that idea. It takes a combination, I think.

———————————————————-

Want to read more stories of open data innovation?

Like Socrata on Facebook.

Follow Socrata on Twitter.

Three months ago, we announced that Socrata had started working on an open source option we call the “Socrata Open Data Server, Community Edition.” This is the core piece of our technology that powers our SaaS platform and an equally important part of our strategy to promote worldwide adoption of open data standards. I blogged about this here and here.

Let’s quickly recap why Socrata has committed to this project.

Our Goals for Community Edition

We’re offering an open source product for a number of reasons, all related to accelerating and broadening the growth of open data. We want to:

- Promote data portability throughout the open data ecosystems.

- Support open source software policies in public organizations around the globe.

- Encourage the development of software on top of open data.

In the words of Kevin Merritt, our CEO, “Socrata is investing in an open source product because it will help us accelerate mainstream adoption of the open data cloud model as the de facto enterprise data architecture. We envision a future where the 99 percent of data still locked up in legacy proprietary systems will be open and accessible to the masses.”

What “Community Edition” Will Offer

The “Socrata Open Data Server, Community Edition” supports the ongoing development of open data standards in three key areas, all required for a thriving ecosystem:

Data Catalog Interoperability – Enables universal federation of different open data catalogs using a standard catalog schema, based on the W3C Data Catalog Vocabulary (DCAT).

Data Portability Based on Standard Data Formats – Standardizes outputs including JSON, XML, and CSV, as well as RDF and other Linked Data standards. The goal is to evolve towards standard schemas that developers can use for popular data sets, based on real-world examples and collaboration between data publishers.

Application Portability Based on Open Data API Standards – Standardizes the Application Programming Interfaces (APIs) used to programmatically access open data, using established paradigms and protocols such as REST, HTTP, and Structured Query Language (SQL).

What Today’s Release Includes

Today, we’re happy to share our initial open source release which, over the coming weeks, will mature into a full open data API server that provides:

- A free and open implementation of our SODA 2.0 API, so anyone concerned with lock-in or proprietary APIs doesn’t have to worry anymore. They can setup the open source version and any third party applications using SODA 2.0 that go against Socrata can go against our open source offering, as well.

- A solution to an interesting problem out there: how to create standard APIs and usage over data that can vary wildly in terms of how often it changes, how big it is, and how often it is accessed.

- Code that is modular and makes it easy for other people to not only contribute code to the core project, but also to make modules for different data stores or functionality.

This initial release will take place over the next three weeks. For more details about the architecture of the “Socrata Open Data Server, Community Edition,” the upcoming phases in our open source roadmap, and information on how to contribute, please visit http://open-source.socrata.com.

To get a good understanding of our new SODA API, if you’re not familiar with it already, look at our documentation on http://dev.socrata.com. Or, for a closer look at the specification and implementation of our SoQL query language, see https://github.com/socrata/soql-reference.

We will keep you up to date on our progress as we complete this first phase. We would love to get your feedback so please stay in touch with us as we release more code and updated documentation.

Will Pugh is Socrata’s Chief Technology Officer and is leading the development of the “Socrata Open Data Server, Community Edition.”

———————————————————-

Like Socrata on Facebook.

Follow Socrata on Twitter.

By Jason Hare, Open Data Program Manager, Raleigh, North Carolina

North Carolina’s first-ever Data Jam will take place at HUB Raleigh on April 22nd, 2013 from 12- 6 pm EDT. Register here.

Who’s invited? Anyone who wants to leverage education, health, and “green” data to create apps to spark a new company, and also benefit society. 

The event starts with a reception, lunch, and an impressive line-up of speakers, including U.S. Chief Technology Officer Todd Park. Once the initial mixing and mingling ends, the real fun begins as participants form groups and plan their apps.

The top four ideas from that initial meeting on the 22nd will be matched to entrepreneurs and working groups. Actual, functioning applications based on those winning ideas will be due on August 1st and the best ones will be selected for a 30-day “accelerator,” providing resources like business coaching and technology.

Finally, we’ll have a Datapalooza Celebration on September 12th, during Triangle Entreprenuership Week. The celebration will feature three keynote speakers, as well as presentations by the competitors and announcement of the winning apps.

It All Started When…

How did this great event get started?

In December, the White House hosted a USAID Data Jam. Until that moment, I only thought of the White House in terms of data.gov. I was not aware of the amazing job The Office of Science and Technology does in promoting the use of open data.

I attended the White House event. Everyone else there was Federal and I was the only municipal person. When I returned to Raleigh, I gave a presentation about my experience to the Raleigh HUB. That’s where the idea for the NC Data Jam got its start.

A Team Effort

Some other folks in the Triangle and I decided that we needed to host a data jam of our own. Planning it was exciting. At times, not knowing exactly who was going to attend or speak, and what the venue would be, was frustrating. What data sets would we have and would our portal be ready on time?

When we started planning for this I had been with the city for nine weeks or so. I did not know many folks within the city but I did know several people at Socrata.

Socrata worked with me to develop a timeline that was realistic. We launched right on time. March 15th was the go-live date for the portal that will house the data jam data. After the launch and the press coverage getting data is much easier. State and county agencies, as well as the surrounding municipalities, are all participating.

This is about our region and not one city. Our municipalities and county have come together to collaborate on a regional data ecosystem. One person did not do this. Dozens of people from all over the Triangle gave their input and time to move the discussion toward cooperation and not competition.

In particular, I want to thank Jason Widen, Mital Patel, Chris Gergen, Erin Monday, and Zach Ambrose for their efforts and enthusiasm. The Regional Data Jam is a celebration of community as much as it is data.