Open Tasking: Exoplanet Data in JSON Format NEEDED

I love programming. And i'm thankful when i heard that today is Ada Lovelace Day. Ada Lovelace is known to be the World’s First Computer Programmer. I was going to do something else today but Lovelace reminded me to code instead. And so I did.

I worked on some exoplanet data today, and have a goal of creating a code-accessible database of exoplanets. But then time flies so fast. I can't do it alone, but I know I belong to a hive of coders and i'm just one node among many of the programming collective. So i'd like to tap the collective and start "Open Tasking". It's like this: i'll tell you where i'm at with this self-inflicted project, and then i'll let you know what kind of help that i need on a particular task. In return, I will share what I learned in the hopes that it will benefit others.

Basically, I am setting up a CouchDB database for Exoplanets. It will be something anyone can use and replicate for any purpose. There's a ton of sub-tasks that need to be done before it becomes a reality so i am posting this as i go along. At the moment, I need help to write a script to convert the XML format of exoplanet data into JSON format so i can import them into CouchDB.
The source data can be found here: Open Exoplanet Catalogue [ https://github.com/hannorein/open_exoplanet_catalogue/tree/master/data ] and the intended destination where the exoplanet JSON will be stored will be in here Exoplanets at Cloudant [ https://cloudant.com/futon/database.html?metapsyche%2Fexoplanets/_all_docs ].
[ If you really want to take a peek at where i'm at right now feel free to check The Exoplanet Viewer. It's really nothing at this point actually, just some preliminary code ]

I've already contacted HannoRein of the Open Exoplanet Catalogue and he said he has no plans to provide the data in JSON format. Bummer. So, right now, I am trying to write javascript code to convert XML to JSON so I can automate a batch conversion of the XML exoplanet data into JSON and then load them into my CouchDB database at cloudant. Why Javascript? So i can use it with node.JS and make streamlined process to keep up with the fast-paced exoplanet updates. So if you already had experience with the task described, and you already have a working set of code, please help me.

I've done quite some research on this and i can't find any usable code at this point. The JSON output should validate at JSONLint [ http://jsonlint.com ]. At the moment, I am using this http://extjs.org.cn/xml2json/xml2json_online.php to convert XML manually. But as I said, I need a streamlined process to keep up with the rapid pace of exoplanet data growth and updates.

Links:
Exoplanets and Open Data
Open Exoplanet Data on CouchDB

Exoplanets and Open Data

Something exciting is brewing behind the scenes between Exoplanets and Open Data. The developer of the excellent Exoplanet App (for iPhone) has started a repository on GitHub for Exoplanet datasets, called The Open Exoplanet Catalogue. The great thing is that this repository is a mix of automated and community-driven movement in terms of keeping up with the fast-paced incoming stream of raw exoplanet data. It opens up an open massive data set (OMDS) for the exoplanet community to tinker with.

The automated part, which interests me a lot--is a python script that pulls in the latest updates from our exoplanet pointman named Jean Schneider from the great Exoplanet Encyclopedia.

Here’s how I can briefly describe the process: Jean Schneider gets new exoplanet information, updates his database. The python “bot” script grabs Jean's newly-updated exoplanet data and re-generates new exoplanet XML files, then puts them to github. You may download and peek at that bot script from here. And most importantly, you can grab the latest exoplanet dataset in all it's XML glory--ready to be consumed by any app or software you can dream of.

For the community-driven part of the setup, anyone can contribute scripts. An example would be for someone to write a derivative of the python script to churn out exoplanet files into JSON format, instead of XML.

It is still on the early stages so at this point "some branches of this repository are updated automatically...and the master branch is updated manually for now, to ensure consistency."

This repository is everything I’ve ever wished for. And I am thankful for it. This would be very useful for the renewed field of Data Visualization, and Data Journalism. This system will truly benefit the community of Open Data Journalists and Researchers within the exoplanet community. On a side note, for all the Android Developers out there: Here’s all the latest data, now please build the Exoplanet App for Android! And for those who can code, please contribute to this endeavor!

"The purpose of the Internet will be to catch the alien mind. It will be coded by human fingers, but it will be truly alien."
~Terence Mckenna

Links:
Python Bot Script
The Open Exoplanet Catalogue at GitHub