Showing posts with label Data. Show all posts
Showing posts with label Data. Show all posts

July 18, 2012

Oh What A Tangled Exoweb...

As everyone in the exoplanet community have noticed, the Exoplanet Encyclopedia (exoplanet.eu) has updated its site with a nifty new look along with new functionalities. It's great. And I love change. The only thing is that it broke my Total Known Exoplanets Chrome Extension. I didn't think anyone ever uses it, until a twitter user inquired on when it will be fixed. So i spent some time to fix it. And now i'm happy to report that the chrome extension is back up online again. Well, at least for now, it's not showing blank. I know it's showing a count that is still late by a factor of weeks but it will be improved on the next deployment. I will now explain the changes that happened under the hood and what it would mean onwards.
When exoplanet.eu changed its pages, my robotic HTML scraper was no longer getting the updated exoplanet count. Much as i tried, I can no longer grab hold of the updated count from that site. So i turned to the coolest open ExoAPI.com to provide the exoplanet count. Using a simple AJAX call, it returns some data that includes the total count. Of course there's always room for improvement and I contacted the awesome developers of ExoAPI to provide a function to grab *just* the total count for speed and lighter load. So now my exoplanet count chrome extension fully depends on ExoAPI to show the latest stats on exoplanets. And I got some new upgrades planned that will make use of ExoAPI some more. Yes, this is another use case showing how important an open exoplanet API is.
Here's some quirky curiousity on how Salaak and Silver Surfer keeps track of planets. I am definitely sure that they maintain a database of planets to do their job (it would be awesome to see at least one issue of green lantern corps mention that 'fact') and that they must have some form of API in order to pass along data and work with other advanced lifeforms' technology. In light of this fiction, can you imagine what it would mean if we suddenly made contact with a more advanced civilization? For some reason, I think that their web developers would be awesome enough as to provide an API for us to be able to access their own planet database. But prior to that, via the Known Worlds Law it is automatic that advanced civilizations have a huge database of planets containing information on gazillions of worlds. And I am optimistic that they will share that information--which would mean access to study trillions of planets!

April 17, 2012

ExoAPI: SpaceApps Challenge

Yay! The ExoAPI I proposed to be included as a challenge to the INTERNATIONAL SPACE APPS CHALLENGE has just been posted. For now I just wanted to get the word out and encourage anyone and everyone to support this endeavor and share their ideas to help create an ExoAPI, a platform that will allow any programmer to grab exoplanet data and create mashups that is accessible on the internet.

The exoplanet era is upon us. And data about exoplanets are growing at a massive pace. However, most of that data are not being harnessed to its full potential. Most of it are locked into private databases that are scattered. The ExoAPI goal will allow us to harness these exoplanet datasets by making it accessible in a streamlined fashion.

Web Programmers like to create mashups and the first thing that must be done is to create accessible datasets. Currently, the most popular format on the web is JSON and the way to go about it is via AJAX or RESTful interface using javascript. If we are able to provide this capability to access exoplanet data in a RESTful way, it would allow web programmers to create fun and interesting things with exoplanet data.

With the exoAPI, programmers will be able to create mashups, games, visualizations, mobile apps, and an interface to allow the public to participate in research and discovery. The possibilities are endless! So please. Join this endeavor and lets move onwards to brave new worlds!


Note: For those who have a background in coding, I would like to encourage you to contribute a few cycles of your brain processing power to a worthy cause that has something to do with space and the future. I encourage you to participate in the International Space Apps Challenge. Please sign up and browse through all the other challenges and look to see where your talents could most likely be helpful.

November 6, 2011

Our Expanding View of the Universe

Recently, I've started collecting Data Visualizations and Infographics about Exoplanets. You may see some of them randomly displayed in the Exoplanetology homepage. I use a third party tool called Vi.sualize.Us to make it quick and easy for me to tag images and share awesome exoplanet visualizations that i come across in the web.

For my latest entry, I'd like to share this nugget of Interactive Data Visualization about our expanding view of "The Visible Universe, Then and Now".

As you marvel at the new astronomical objects we have discovered, don't forget to compare how it looked like back then in the 1950's and how little we knew about the things around our star.

Exoplanets play a big part in the recent upgrades in how we see our universe. In just a mere decade, an entire generation of humans would realize that our earth is just one among billions and gazillions of planets out there.

Take a moment to wonder how much we have learned in such a short span of time. Was there any difference at all in how we behaved as a species? More importantly, take a moment to look into yourself and see how the new knowledge of other worlds changes your way of thinking and how you now see the world.

October 19, 2011

The Total Known Exoplanets Google Chrome Browser Extension

Just a quick note to tell you something about the first ever Google Chrome Browser Extension I ever made which is a very simple counter for the total known exoplanets. With a click of a button, you'll see how many known exoplanets humanity has discovered so far.

So if you use the Google Chrome browser a lot, and if you’re obsessed about exoplanets, why don’t you install this handy extension so you can feed your exoplanet fix? You can install it from the official extension page from Google: Total Known Exoplanets Chrome Extension.

So how does it work? Well, everytime you click on the icon, it sends me an alert, and I type the number of the latest count of known exoplanets on that tiny bubble. Simple enough, right?

You’d think i’m crazy to do that. And you’re right. That’s why I automated the process. So here’s what happens behind the scenes after your click. (Warning: technical stuff ahead.)

IMPORTANT UPDATE (July 18, 2012) !!!

After you click on the icon, the extension requests data from a pipe i made on Yahoo Pipes, called "Exoplanet Count Spy" which spews out data in JSON format (which contains the updated number of known exoplanets).

Yahoo Pipes allows you to get hold of data in convenient JSON format from an external RSS feed source. Its JSONP capability is a very useful feature, which subverts the cross-domain barrier and allows you get data from external sites and use them.

So where does my “Exoplanet Count Spy” pipe get its data from? It grabs it from a news feed from Feed43, another great third-party service on the web that generates RSS feeds from an HTML page. Now comes the real source of data where i get the number from.

It's been a long chain of events so far only to find out that the ultimate source of data is the Exoplanet Catalog from the venerable Extrasolar Planets Encyclopaedia (exoplanet.eu) maintained by Jean Schneider. Many thanks to him for faithfully updating the tally of exoplanets. Without him doing it, this extension will lose it's functionality.

All those things described happen behind the scenes of the extension. But on the surface, the Total Known Exoplanets counter simply consults the exoplanet tally page from the Exoplanet Encyclopedia and shows you the result on your browser. That’s all.

I hope you find the Total Known Exoplanets extension useful!

October 7, 2011

Open Tasking: Exoplanet Data in JSON Format NEEDED

I love programming. And i'm thankful when i heard that today is Ada Lovelace Day. Ada Lovelace is known to be the World’s First Computer Programmer. I was going to do something else today but Lovelace reminded me to code instead. And so I did.

I worked on some exoplanet data today, and have a goal of creating a code-accessible database of exoplanets. But then time flies so fast. I can't do it alone, but I know I belong to a hive of coders and i'm just one node among many of the programming collective. So i'd like to tap the collective and start "Open Tasking". It's like this: i'll tell you where i'm at with this self-inflicted project, and then i'll let you know what kind of help that i need on a particular task. In return, I will share what I learned in the hopes that it will benefit others.

Basically, I am setting up a CouchDB database for Exoplanets. It will be something anyone can use and replicate for any purpose. There's a ton of sub-tasks that need to be done before it becomes a reality so i am posting this as i go along. At the moment, I need help to write a script to convert the XML format of exoplanet data into JSON format so i can import them into CouchDB.
The source data can be found here: Open Exoplanet Catalogue [ https://github.com/hannorein/open_exoplanet_catalogue/tree/master/data ] and the intended destination where the exoplanet JSON will be stored will be in here Exoplanets at Cloudant [ https://cloudant.com/futon/database.html?metapsyche%2Fexoplanets/_all_docs ].
[ If you really want to take a peek at where i'm at right now feel free to check The Exoplanet Viewer. It's really nothing at this point actually, just some preliminary code ]

I've already contacted HannoRein of the Open Exoplanet Catalogue and he said he has no plans to provide the data in JSON format. Bummer. So, right now, I am trying to write javascript code to convert XML to JSON so I can automate a batch conversion of the XML exoplanet data into JSON and then load them into my CouchDB database at cloudant. Why Javascript? So i can use it with node.JS and make streamlined process to keep up with the fast-paced exoplanet updates. So if you already had experience with the task described, and you already have a working set of code, please help me.

I've done quite some research on this and i can't find any usable code at this point. The JSON output should validate at JSONLint [ http://jsonlint.com ]. At the moment, I am using this http://extjs.org.cn/xml2json/xml2json_online.php to convert XML manually. But as I said, I need a streamlined process to keep up with the rapid pace of exoplanet data growth and updates.

Links:
Exoplanets and Open Data
Open Exoplanet Data on CouchDB

August 16, 2011

Open Exoplanet Data on CouchDB

I haven’t been able to do much coding lately. Summer has been busy and my laptop broke. There’s so much I want to do for the field of exoplanetology but i just couldn’t even start without a usable open database platform.

Anything practically useful begins with data and the truth is, we have a great amount of exoplanet data floating around already in all these exoplanet catalogs, but they’re just lying there, and not really being maximized to their full potential.

How can we enable the the wider use of exoplanet data in more useful and practical applications? It begins by providing an open access to that data and a programmatic way of accessing all that exoplanet data.

CouchDB has been my choice due to its advantage of replication and flexibility. I cannot even begin to list down all the many wonderful advantages of using couchDB for an open exoplanet data source that anyone can build upon.

Exoplanet data that is inside a couchDB database can be used and re-used via typical web standards, primarilly javascript. It spews out data in JSON format and this allows anyone to create engaging websites, games, and apps atop all that data. I am even positive that couchDB can be used for scientific analysis of a huge exoplanet data to find patterns and trends, and create dynamic visualizations from it.

There doesn’t seem to be any couchDB implementation of exoplanets yet, so I am posting this to start off the movement in this area of exoplanetology. Watch out for this space.

March 3, 2011

The Exoscope

We are visual creatures. We want to understand things from the context of a larger view. And that is true with all the exoplanets we have known so far. How can you relate with these amazing orbs in a more direct way? When you look up in the night sky, where are the exoplanets?

{Drum roll}...The Exoscope.

The Exoscope is a web-based tool to enable you to see the locations of exoplanets in the sky. It's a tool for Exogazing, the activity of spotting stars with known exoplanets.

Exoscope is actually a spin of the awesome Chromoscope. The ability to change and blend the background to different wavelengths gives that dramatic feeling, and that is why I love Chromoscope so much!

The excellent developer, Stuart Lowe (@astronomyblog) and others did all the work to bring you that awesome goodness of chromoscope. I simply imported an exoplanet KML file, and the chromoscope engine did all the rest of putting the markers for the locations of the stars with known exoplanets.

Really, I didn’t do much. I just downloaded the chromoscope application (which is freely available) and added two lines of code to import the exoplanet KML and show the constellations by default. And that was it!

The hard work is actually up ahead. And that is how to always keep the exoplanet KML file updated.

I need help and ideas on how to generate the exoplanet KML file from the most updated source of exoplanet data, The Exoplanet Encyclopaedia. Also, I need some script to generate KML files from a CouchDB couchapp. So, if anyone can give leads on that regard, please ping me. For now, some info i can provide for whoever wants to take up the challenge is this: The developer of the most excellent exoplanet app for the iPhone/iPad (@exoplanetapp) has provided a digestible format of Jean Schneider’s exoplanet data, via The Open Exoplanet Catalogue. So, all I need is for someone to generate JSON data in addition to the XML files. I seriously need JSON so I can import them into my CouchDB exoplanet database app (which is also another work in progress).

Other tasks on the pipeline for this mini-project are:
1) How to center the field of view (FOV) on the exoplanet via a link from other sources (such as The Exoplanet Seeker). I like how WikiSky does it so i hope to implement the same thing.
2) How to host the image files into my own Amazon S3 account to ease the burden from Chromoscope.
3) How to add other stunning backgrounds
4) Cosmetic styling, and UX improvement for each exoplanet popup window (and how to pipe in data dynamically)

That’s it for now. I Hope that even while it’s still in progress, you’ll enjoy the stunning exoplanet landscape from The Exoscope!

Links:
Chromoscope
The Exoscope
Exogazing
The Open Exoplanet Catalogue
Exoplanet Encyclopedia

November 20, 2010

Exoplanet 500

I wrote this post to mark a milestone in Exoplanet Science. There are now over five hundred known exoplanets in our database.

It may not be such a big deal. Five Hundred is not much. Not until one considers the fact that gazillions of planets wander the galaxy. It means that cataloguing worlds is an infinite pursuit. Such task tickles the potential of the whole human race, and the human mind.

We have "Indy 500", and we have "Fortune 500", now why not have "Exoplanet 500"?

Yes, "Five Hundred" is fun to utter. And as I sing it out loud, I reflect upon the accomplishments of the Astronomers whose dedication continue to expand our view of the Cosmos.

Amazing how far we've gone as a curious species.

Now's a good time to pause and appreciate the contributions of the rest of the Scientists, Philosophers, Artists, Thinkers and Dreamers--not only those that are currently in our midst, but especially those who have gone before us. They are the Giants upon whose shoulders we are standing on to get a better view of our wondrous universe, enabling us to glimpse New Worlds.

"For I would walk 500 miles, and i will walk 500 more..."
~ The Proclaimers, I'm Gonna Be (500 Miles)

Tadalala! Tadalala...tadalalalalalaa...


Links:
Exoplanet Encyclopaedia

November 4, 2010

My First Foray into Data Visualization with Exoplanets

Einstein said, "If I can't picture it, I can't understand it." This statement proved itself to be true on my first foray into Data Visualization. And let me add that it was absolutely a fun experience being able to play around with all the data using the excellent tool provided by "Many Eyes" which was instrumental in helping to reveal insights that lay hidden within those rows and columns of numbers. And of course, it wouldn't be possible at all without the data! So I am thankful for the folks at exoplanets.org for providing the dataset.

Now I realized that, "If you can't explain it simply, you don't understand it well enough." Yes, Einstein also said that. And I think that is the final part of Data Visualization (which i presume overlaps with Data Journalism in some way, too). It was fun generating the visualizations, and analyzing them to find insights was challenging--but it is not complete without the ultimate goal of communicating what I have discovered.

Sure, the pictures speak for themselves (in more than a thousand words), but there is an added benefit in formulating a simple statement to convey the insight: it makes you learn the concept even more. Finally, being able to share that understanding completes the joy.

Let me warn you that I am not a scientist, so please bear with me as i briefly explain my own original findings from these pictures (and please correct me if i'm wrong). Also let me iterate that I'm not a designer or an artist, so the graphics presented in this post are no match compared to the aesthetics of David McCandless (Visual Miscellaneum), and the analytical prowess of Edward Tufte (Envisioning Information).

The conclusion(s) I've drawn from the visuals pretty much explains it concisely. The additional notes are supplementary.



Fig. 1: Visual Magnitude vs Discovery Method (used to find the exoplanet)
Conclusion: The magnitude or brightness of host stars whose planets were discovered via Transit Method are generally dimmer than those stars whose planets were discovered via Radial Velocity (RV) Method. Here's another proof.
Analysis:
Transit Method works best on dimmer stars because it does not drown out its own planets in its glare as much as a bright star would. Radial Velocity (RV) method works best on brighter stars for better spectographic analysis.

Fig. 1: Visual Magnitude vs Discovery Method

Notes:The label delineating the two methods are not noticeable so i need to point out that the left section are those host stars with planets were discovered via RV method. The smaller section on the right with a noticeably darker shade are those stars with transiting planets.
The dimmer tint for high values of Visual Magnitude (V) fits well with the fact that stars with a higher value of magnitude (V) are actually dimmer than those with a lower value.
[Link to the interactive version at Many Eyes]


Fig. 2: Discovery Method vs. Exoplanet Names
Conclusion: Exoplanets discovered via RV method have more "generic" sounding names than those discovered via transit method.
Analysis:
Here's a nice tip on how to quickly guess whether an exoplanet was discovered via RV or transit method: If it's generic-sounding, or if it bears the host star's name (like "HD blah-blah")--then chances are, it’s discovered via Radial Velocity!

Fig. 2: Discovery Method vs. Exoplanet Names
Notes: Exoplanet names are commonly derived from the host star name plus the alphabetical index of the planet by order of its discovery date. Often, the instrument used to discover it are then used in lieu of the star name (for example: Kepler-4 b, or CoRoT-7b)

[Link to the interactive version at Many Eyes]



These are only some of the visualizations I came up with while playing with the exoplanet dataset for around an hour or so. There are definitely many more correlations you can uncover via the other methods at Many Eyes or other tools. Make sure you also play around with the data and let other people know about the insights you come up with. Head over to Many Eyes and use the same data I uploaded to generate your own visualizations.

Disclaimer:
1) These graphs and visualizations are what we can statistically glean from, given the current data provided. The trends I pointed out in this post will change in the next few years. For example: When Kepler starts to announce their discoveries, the number of exoplanets discovered via Transit Method will suddenly skyrocket and overtake the total number of exoplanets discovered via Radial Velocity (RV). (It will be interesting how to visualize that trend when we add the function of time.)
2) Also, these visualizations does not take into account those exoplanets discovered by a mash-up of two or more methods (such as those exoplanets discovered by using Transit method in tandem with RV).

Links:
Many Eyes
Dataset from Exoplanets.org

October 22, 2010

Exoplanets and Open Data

Something exciting is brewing behind the scenes between Exoplanets and Open Data. The developer of the excellent Exoplanet App (for iPhone) has started a repository on GitHub for Exoplanet datasets, called The Open Exoplanet Catalogue. The great thing is that this repository is a mix of automated and community-driven movement in terms of keeping up with the fast-paced incoming stream of raw exoplanet data. It opens up an open massive data set (OMDS) for the exoplanet community to tinker with.

The automated part, which interests me a lot--is a python script that pulls in the latest updates from our exoplanet pointman named Jean Schneider from the great Exoplanet Encyclopedia.

Here’s how I can briefly describe the process: Jean Schneider gets new exoplanet information, updates his database. The python “bot” script grabs Jean's newly-updated exoplanet data and re-generates new exoplanet XML files, then puts them to github. You may download and peek at that bot script from here. And most importantly, you can grab the latest exoplanet dataset in all it's XML glory--ready to be consumed by any app or software you can dream of.

For the community-driven part of the setup, anyone can contribute scripts. An example would be for someone to write a derivative of the python script to churn out exoplanet files into JSON format, instead of XML.

It is still on the early stages so at this point "some branches of this repository are updated automatically...and the master branch is updated manually for now, to ensure consistency."

This repository is everything I’ve ever wished for. And I am thankful for it. This would be very useful for the renewed field of Data Visualization, and Data Journalism. This system will truly benefit the community of Open Data Journalists and Researchers within the exoplanet community. On a side note, for all the Android Developers out there: Here’s all the latest data, now please build the Exoplanet App for Android! And for those who can code, please contribute to this endeavor!

"The purpose of the Internet will be to catch the alien mind. It will be coded by human fingers, but it will be truly alien."
~Terence Mckenna

Links:
Python Bot Script
The Open Exoplanet Catalogue at GitHub

September 2, 2010

Musings on the Implications of Synthetic Life on Astrobiology

A year before Craig Venter announced the successful creation of the first synthetic lifeform, i asked him what insights it could bring to the science of Astrobiology. I asked what it would mean if synthetic lifeforms gave off different biosignatures and how that would be beneficial to those curious about life on other worlds.

I don’t think i got a concrete answer from Craig Venter at that time, and even after that, I’ve never really read any writeup about the tremendous impact synthetic life will bring to Astrobiology. That was then during the Crossroads conference, way back in May 2009 where Dimitar Sasselov was present as well.

I simply headed home that night with a glimpse of the ground-breaking event that lay ahead.

A year later came Venter's announcement of the first ever synthetic lifeform. Now Fast forward to July 2010, Sasselov suddenly mentions Synthetic Life in his controversial TED talk. But everyone was simply distracted by the controversy and confusion introduced by Dimitar when he used the word "earth-like" rather than "earth-sized". Perhaps many missed another big picture--which is the link between Synthetic Biology and Astrobiology and the impact it would bring to our culture and future.

Now imagine this scenario:

Consider a planet, and know it’s properties, such as surface gravity, atmospheric composition, pressure and chemical characteristics, temperature, and so on. Then create a model of that environment, or a subset of it--called a “hab bubble”. Virtually design an extremophile microbe that would survive within that bubble. Simulate the microbe and it's habitable bubble in a "virtual world". Send that simulator on-board a probe, continuously adjusting the parameters as the probe travels toward an exoplanet. Feed the simulator with updated data as new findings are discovered along the way. Upon reaching the destination, manufacture the microbe using local resources or chemicals on that planet, "build and live off the land". Then unleash the lifeform by setting it loose on the surface of the exoplanet.

This simple scenario became the inspiration for a short story, my first #fridayflash or #FlashFiction entitled “The Biosynthe”.

Links:
The Biosynthe (Short Story)
Craig Venter's Synthetic Lifeform
Dimitar Sasselov's TED Talk

August 19, 2010

Possible Data Mining Age in the Future of Exoplanet Science

Exoplanets. Distant worlds light-years away, yet they continue to move ever closer to humanity’s psyche, enticing us to declare them worthy of serious research.

The study of exoplanets is of such enormous intellectual importance and it helps us understand our place in the universe,” said Louis Friedman of the Planetary Society.

The search for exoplanets is one of the most exciting subjects in all of astronomy,” the Astronomy Decadal Survey committee expressed their enthusiasm in a report entitled New Worlds, New Horizons in Astronomy and Astrophysics.

Thus, a program was recommended “to explore the diversity and properties of planetary systems around other stars, and to prepare for the long-term goal of discovering and investigating nearby, habitable planets." The survey proposed The Wide-Field Infrared Survey Telescope (WFIRST), a $1.6 billion dollar telescope that would provide us with more exoplanet data. The report states that “in addition to determining just the planetary statistics, a critical element of the committee’s exoplanet strategy is to continue to build the inventory of planetary systems around specific nearby stars.

Now look closer at those words. “Inventory”, “census” and “statistics” of planetary systems. Clearly, the current focus is to continue to “gather” more and more data. We just turned into proverbial planet hunter-gatherers!

Kiddding aside, the excitement to gather more exoplanets in our database is great. But then I feel that something is missing, and that something must be done in addition to it all.

I feel that the future prospect of “processing” must also be planned in parallel with the construction of these telescopes.

We are continously building instruments to gather more and more exoplanet data but we are not preparing the infrastructure to handle the imminent data explosion in Exoplanetary Science.

I anticipate a bottleneck when all these planet-hunting telescopes finally get deployed and the copious amounts of data comes flooding in. On what basis am i saying this? Well, take this for example: Dozens of Earth-mass planets could already be within the database of the Kepler Team right now even as we sit waiting for their announcement. That is why i think that not just one earth-sized planets would be announced at one time, but several of them. It’s just that the Kepler scientists are extra careful about their greatest discovery ever in humanity's history. They are simply making sure that their discoveries are solid before they reveal it to the whole world.

But perhaps it’s understandable to withhold data about historical “firsts” because so much is at stake for the Scientists making the announcement.

The raw data will eventually be shared to the public for Citizen Scientists to gobble up. But I can't help but think that in some way, it’s a mini-bottleneck. And looking ahead, there is no infrastructure for outside help to “harvest” planets from the raw data the will eventually pour in. So far so good, only pure Scientists can harvest the goods.

In my previous post, I asked these questions: How will Scientists keep up with petabytes of raw data? How will web technology keep up? How will the internet enable Citizens to contribute to science for the love of it? What needs to be done?

Looking around at the changing face of Science that results from the enabling power of web technology, we see the important contributions of a swarm of minds working together for Scientific goals.

Take a look at GalaxyZoo, FoldIt, StarDust@Home, Rosetta@Home, Einstein@Home and SETI@Home. These are awesome computational engines that tap the human potential in analyzing massive amounts of data, to produce novel scientific discoveries. Despite the public excitement in the field of exoplanets, I'm surprised that there is no program like them that are focused on exoplanetary science. I have reason to think that in some way, the same idea of "distributed processing" can be applied to the deluge of exoplanet data that will arrive in the coming decades.

"We're at the dawn of a new era, in which computation between humans and machines is being mixed," says Michael Kearns--a computer scientist dealing with the concept of distributed thinking.

So how can we apply the power of distributed thinking in Exoplanetary Science? Who must jump-start the GalaxyZoo for Exoplanets? Should NASA or ESA do it? Should ExoPAG work on it? Universities? Or should the private sector do it? Should someone write a grant proposal? Or perhaps open a kickstarter project for it?

It’s been said that we are currently in the Golden Age of planetary discovery. But if we don’t begin to create some way to collectively "process" the huge amount of data gathered by planet-hunting telescopes, a “bottleneck era” might follow. It will be an era in which we are virtually “aware” that planets lie hidden within massive amounts of data stored somewhere in hard drives locked away at some facility. And like the exoplanet that was re-discovered within the Hubble archives, it would take years to mine these planets from out of the digital ones and zeros and be announced as a belated exoplanet discovery.

I am guessing that that such an era may likely occur if we don't build a system to process the data in a collective fashion. But then we would refuse to call it a bottleneck era. We would simply name it a much better-sounding term, The Data Mining Age of Exoplanetary Science.


Links:
New Worlds, New Horizons in Astronomy and Astrophysics: http://www.nap.edu/catalog.php?record_id=12951
$1.6 billion telescope would seek out alien planets: http://www.msnbc.msn.com/id/38692359/ns/technology_and_science-space/
Dark Energy and Exoplanets Top List of Astronomy Priorities: http://www.wired.com/wiredscience/2010/08/nrc-space-report/
Citizen science, People power:
http://www.nature.com/news/2010/100804/full/466685a.html

August 14, 2010

The Exoplanet Seeker and Mendeley

I recently built a tunnel to a new source of information about exoplanets. This new source is an outfit called Mendeley, in which i tapped their API to bring in lists of Scientific papers about exoplanets and present them inside The Exoplanet Seeker's pages.

There’s still a lot to be improved in The Exoplanet Seeker, especially in terms of getting the right names of exoplanets. Often, a simple blank space causes null results (HAT-P-7b vs. HAT-P-7 b) from the datasources. So, i would continue to improve it as time goes by. For now, the most notable upgrade is the addition of Mendeley among the data sources, and the creation of a faster launcher page for The Exoplanet Seeker. It now has a new home, so please update your bookmarks.

I heard about Mendeley in the past, but i felt it was too “sciencey” and inaccessible for a laymen like me. All that changed when I heard about their open API. So i immediately coded up something to test what they had to offer. And it did not disappoint.

In as much as new exoplanet discoveries are starting to change the landscape of Humanity’s Thought (to be punctuated by the revelation of earth-like planets) the evolution of the web is also changing the landscape of Science. What Mendeley has done with their Open API is great for the proliferation of Scientific study via the internet for both professional scientists, researchers and lay-people.

What i am anticipating is the imminent data explosion in Planetary Science brought about by the accelerating discovery of new exoplanets. How will Scientists keep up with petabytes of raw data? How will web technology keep up? How will the internet enable Citizens to contribute to science for the love of it?

Clearly, we are living at a time like no other. Right now i can only say two words that keep me excited about the changing landscape of the relationship between Science and Technology: Fascination and Wonder!

Links:
Pipe Dream: http://exoplanetology.blogspot.com/2010/01/data-pipe-dream.html
Space Matters: http://exoplanetology.blogspot.com/2008/06/space-matters.html
The Exoplanet Seeker Home: http://bit.ly/ExoplanetSeeker

July 29, 2010

Exoplanet Seeker and Wolfram|Alpha

Just a little geekery and heads-up for those who are so inclined. I've just managed to hack up a link to Wolfram|Alpha's data set from my crazy ol' Exoplanet Seeker thingie.

But wait! There’s something else goin' on here: since The Exoplanet Seeker’s auto-suggest feature is powered by Freebase, I’ve actually married Freebase and Wolfram|Alpha together! Although in a primitive way, I’ve proven that it is indeed possible to make them work together. (Hopefully, this gives out a message to both companies about a dream of mine--which is that Freebase and Wolfram|Alpha work together for the good of the interwebs.)

Now enough with the bragging and pipedreams. Let’s get down to business and to the revelation of the secret on how it was all done.

First, let me tell you about Wolfram|Alpha’s cool new feature: W|A Widgets! So I built an exoplanet widget (shown below) using W|A’s widget builder, and piped in the exoplanet name into the URL of the exoplanet info widget. That’s was all there is to it!

But by no means is it perfect. There are lots of data holes. Freebase is not complete with all the exoplanet names, thus it can only suggest what it “knows”. And even if Freebase “knows” an exoplanet name, Wolfram|Alpha often doesn’t know it yet (W|A is slower in terms of updating exoplanet data). And so, selecting the “Wolfram|Alpha” on the dropdown and then clicking “seek” will yield a “not valid input” on the popup window. Of course, all this will improve soon enough. And I appreciate Mr. Wolfram putting up a page specific to Exoplanets.

The field of exoplanet science is progressing so rapidly that there will come a time when a need for exoplanet informatics is needed, if an Open Citizen Science on exoplanet research is ever set up to deal with the data aspect of exoplanets. For now, i hope this serves as a quick reference for exoplanet enthusiasts out there.

By the way, there are cool astronomy widgets in W|A’s gallery. Two of which are shown below. Do check them out! And of course, you must build your own widget!












Links:
Astronomy Category in Wolfram|Alpha Widgets Directory
Wolfram|Alpha Exoplanet Page

July 16, 2010

On Google, Metaweb and Freebase

"We shape our tools, and thereafter our tools shape us."
~ Marshall McLuhan

I thought i'd give you a heads-up on some developments behind the "informatics" of exoplanetology, at least about the tool or platform that I've been using for the past few years.
You may have heard about Freebase, the open database where I add bits of information about exoplanets. Freebase is also the engine behind the auto-suggest feature of The Exoplanet Seeker.
Well, some googly eyes from Google have noticed Freebase and decided to buy it. Metaweb, the outfit behind Freebase is now officially part of Google.
As you know, Exoplanetology has been on Freebase for more than 3 years now. I've enjoyed adding all those bits of data into Freebase, but sometimes I worry that all that time I spend inputting data into Freebase might go down the drain if Metaweb fails to sustain itself. But I kept at it anyways, because its fun contributing what I learn (and learning as I contribute). So now, that worry is alleviated by the fact that Google's vision about the Semantic Web aligns closely with Metaweb's. What makes me happy is that Google and Metaweb will continue to maintain Freebase as a free and open database for the world. Well, that deserves a big "Yay!"

June 15, 2010

On Kepler Data and Beyond

It's almost a dream come true. For so many months, I've been wishing to take a look at actual Kepler data, and I often tweeted how Citizen Science could help analyze exoplanet data. And now that moment is close at hand!
For starters, when you look into this arXiv paper you'll see some data on 306 of the 706 exoplanet candidates.
Also, an important tidbit is that the paper lists 5 multi-planetary transiting systems. If any one among them is confirmed, then it will be the first case of a transiting multi-planet system known.
I was patiently monitoring the MAST website where the promised new data set can be downloaded from, but it didn't come as of yet. In my boredom, I just downloaded the FITS file of Kepler-4, hoping to practice with it before handling the exciting new data.
When I finally got hold of the FITS files via FTP from the MAST website, my initial reaction was: "So now I have the data! But What Next?!"
It was a sudden realization: Yes it is very good that the exoplanet data is being shared to the public, but now we need tutorials on how to analyze those data. Suddenly, I am now wishing that Scientists and Astronomers would post how-to's and tutorials on how to analyze FITS data to find planets! And I am hopeful, that day will come!
For now, I scrambled my way in the dark on finding the right software to be able to view and edit FITS files. Once I found the excellent "FV", I played with the data set and it felt nice seeing regular dips in the curve. So I am thankful for all the people responsible for giving us this chance to see actual scientific data.
Looking at all this, I realized that we are now at a threshold moment where ordinary people can do Science. This is definitely the beginning of a new era of Scientific Discovery. And I say, it's amazing! I can't wait to see what's beyond!!!

Links:
Abstract: http://lanl.arxiv.org/abs/1006.2799 
PDF (Direct Link) http://lanl.arxiv.org/ftp/arxiv/papers/1006/1006.2799.pdf

February 4, 2010

On Resolving Exoplanet Nicknames

While i was building the "Exoplanet Seeker", i discovered a wonderful capability of Freebase and i think i've found an extended use for it's auto-suggest feature.

If you start typing the nicknames of some exoplanets such as "Bellerophon" or "Osiris" or "Methuselah", you would notice that it suggests the correct exoplanet and shows it's official name. For example, if you started typing "Bellerophon" in the empty field it would start showing 51 Pegasi b, the historic exoplanet co-discovered by Michel Mayor. Or typing in "Methuselah" would bring up PSR B1620-26 b--the oldest known exoplanet believed to be 12.7 billion years old.

Immediately i appreciated this capability to bring up the official name by just typing the nickname, and we owe it to Freebase. It actually would prove to be a nice platform upon which to implement the proposal of Wladimir Lyra to give each exoplanet a nickname--in which case the "Exoplanet Seeker" would function as a nickname-resolver. A practical use would be this: If you can't remember what the cryptic official name of that odd exoplanet was, but you remember it's nickname (which often is the case), just go to The Exoplanet Seeker to help you find out the official name!

All this would only work if we saved the nickname in the 'alias' of the exoplanet data within Freebase. Somehow, behind the scenes the system looks up the aliases of objects within freebase and gets resolved by the auto-suggestion capability.

Of course, this is just the tip of the iceberg--many more useful apps can be developed if enough exoplanet data is entered into Freebase. I hope some folks join me in filling up more data about exoplanets on freebase so many other developers can create more mash-ups utilizing this impressive platform.

January 22, 2010

A Data Pipe Dream

I have a dream, a data pipe dream. I'd like to share a vision on how to maximize the usefulness of data in the field of Exoplanet Science.

Even before I built the so-called Exoplanet Seeker, I had something else in mind. I wanted to build a "mash-up" from the treasure trove of data available about exoplanets. Much data on exoplanets are freely available but there is no standard as to how they are shared, and have no structure as to be usable by other web applications. And there is no way to channel them to other applications. In other words there are no "pipes" to send them through in order to make them more useful.

I am thankful that exoplanet researchers are sharing their data online. But now we must move to the next logical step. Most of the data providers do not expose their data in a structured way like XML or JSON. So far, the only one I saw that is friendly to programmers is LookUP, as they share data in XML, JSON, and AVM format.

Currently, there's no way for web developers to create compelling Web Apps or "mashups" from the data available at The Exoplanet Encyclopedia, NStED, or Exoplanets.org. The best I could do was to come up with a very simple exoplanet seeker, in the hopes that it signals a vision of what is to come in the next few years as more exoplanet discoveries pour in.

Think of the hundreds of thousands of exoplanets that will fill scattered databases across the earth. Think of how many Petabytes of data all these discoveries will mount to. And finally, think of how wonderful it would be if all the exoplanet researchers and planet-hunters in the world shared their data in the form that is usable by other Scientists, and open to Citizens as well.

I believe that when an interoperability standard is developed (perhaps something like an "Exoplanet Markup Language", ExoML), or when exoplanet data sources started sharing their data semantically, then a new form of Science will emerge from this abundance of structured Exoplanetary Science data. Awesome things can emerge from this information pipe infrastructure. Perhaps even some new and novel discoveries can come out of it.

If programmers can readily create visualizations from open data, or run simulations using structured data, or create cross references from different planetary characteristics along with their host stars' qualities then perhaps patterns can be discovered, like how they discovered the relationship between the lithium content of stars and the presence of planets around them. I'm sure there are other clues that are hidden in all these data, just waiting to be mined. Even old data can still contain yet undiscovered treasures, like how an exoplanet was found hidden within Hubble's old data archives.

This data pipe dream must have been apparent in two of my previous posts, "The Extremophile Zone" that hints some aspect of this data infrastructure (injecting how social media can play a role in science), and another is "Life is a Pattern", a flash fiction that gives a glimpse of this vision in a metaphorical narrative.

I hope that some organization picks up on this thread to enable a collaborative data pipeline among exoplanet researchers around the world.

January 16, 2010

The Exoplanet Seeker

Today, I just unveiled a convenient way for exoplanet researchers and enthusiasts to "seek" data about specific exoplanets from highly-targeted sources. I call it "The Exoplanet Seeker". It's a simple interface to enable anyone to easily query numerous exoplanet databases on the web to find data on an exoplanet of choice. Currently, it can be found at the homepage at http://www.exoplanetology.com for easy access.

Current exoplanet data sources include the Extrasolar Planet Encyclopedia, NASA's PlanetQuest New Worlds Atlas, The Planetary Society's Catalog, Exoplanets.org's table, ArXiv, and so on.

As you can see, each database has it's own unique focus and presentation. For example, The Planetary Society's Catalogue has orbital diagrams, while exoplanets.org has the lightcurves, and Planet Quest's New Worlds Atlas has the nice graphical charts.

So the Exoplanet Seeker acts as a junction point to all these great sources of exoplanet data. More sources will be added as they come along. And of course, more exoplanets will appear in the auto-suggest list as they get entered into Freebase which powers the auto-suggest feature.

And the great thing about all this is that anyone can contribute by adding data into Freebase--which is easy: just copy the data from the reputable sources! [eyebrows rising sharply? tee-hee] Perhaps this is just the beginning of "Open Source Exoplanetology" via Citizen Science.

The interface that I provided is just a tip of the possibilities if enough exoplanet data is entered on Freebase. Many exoplanet apps can be made from the wealth of free structured data on the Freebase platform.

To use the Exoplanet Seeker, simply start typing into the empty field and a list of exoplanet suggestions will pop up so you can choose accordingly. Then select an exoplanet data repository to search from. Finally, click the "SEEKER" button to search data about that exoplanet from the source database that you've chosen. A new browser window will be opened showing the search results from the external website providing the exoplanet data.

Please enjoy The Exoplanet Seeker and I welcome your comments, and will dodge violent reactions. Do let me know any of your suggestions.

Happy Exoplanet Seeking!

July 23, 2009

Farthest, Nearest...and any exoplanet in between

ExoplanetOn twitter, someone asked me a question how far away the most distant confirmed exoplanet is. The answer is quite tricky. Rapid developments in exoplanet discovery renders the answer outdated within a few months or few years. Therefore, my answer will involve *how* to obtain the answer from the web so that the answer will have a longer "shelf-life" and stay as close to accurate as possible. Hopefully, this post will also benefit those who want answers to related questions--such as what the closest exoplanet is, and so on.
Start by going to The Extrasolar Planets Encyclopaedia and sort the catalog via distance (by clicking the distance column).
The answer yields the topnotch most distant exoplanets to date OGLE-05-390L b, MOA-2007-BLG-400-L b at around 6,500 & 6,000 parsecs. This is roughly around ~21,190 & ~19,500 light-years away respectively.
But remember, independent exoplanet researchers from different countries may publish papers with different values. And you may stumble upon outdated information. The exoplanet SWEEPS-10 is still listed at 22,000 light-years away. So a little bit of research is definitely needed to answer questions in the nascent field of Exoplanetology.
Now if you want to be adventurous and ask a "mashup" question such as "What is the closest Super-earth?" then I recommend playing with the list of exoplanets on Exoplanetology at Freebase.
In that interface, you can add columns and refine the view to obtain details you've never seen before. More details will be reserved for another blogpost about Freebase, but the main thing to note is that answering your questions means getting involved with it in some way.
In a new field of Science, you most often need to input reliable data in order to obtain reliable answers. You can start by getting an account at Freebase and start inputting reliable data from reliable sources.
In this day and age where Citizen Science is proliferating, your interest is very precious. You are encouraged to contribute and learn. That makes you part of the revolution in Science.