Suggestion: API

[coffeemonk]

I know i've made this suggestion a couple times via email, but I thought perhaps putting it on the forum might increase the chances of a response and/or initiate some discussion on the idea.

Basically, I love the idea of a socially driven online comic index with collection tracking, but I equally love having my data available offline and searchable using a separate comic database program (in my case, ROW).

The author of ROW is aware of comicbookdb.com, and already has the ability to scrape data from other online databases (namely, the GCD); so scraping data from comicbookdb.com wouldn't be a huge stretch for the program already. However, given the growth in open online APIs in the last few years, I thought it would really open up a powerful option if this site were to offer an API whereby desktop programs could, at the very least, access issue data and collection management functions.

What I envision is the ability to enter my comicbookdb.com credentials into ROW, and use ROW to synchronize my online collection with the offline database on my desktop machine.

Providing an API would make this super-easy, and would also open up the data channel to other programs and online sites. We could create WordPress, TypePad, Konfabulator, Apple Desktop, Google Desktop or other widgets to display our collection data (subsets or stats) where ever we wanted, providing further exposure to comicbookdb, and links back into the system.

Eventually, as the API expands, the possibility might be opened for issue information in the online index to be added or updated by offline users during synchronization. Trusted users with established history of adding valid, accurate data could be given this option, while leaving un-trusted/new users only with the ability to read the data into their personal databases, and add issues (by reference) to their online collections.

I think the first step would be allowing data access via API, with collection management functions following that.

I'd love to hear Chris' response to this idea, as well as what the greater community thinks.


thanks for listening, and thanks for the great resource!

m@

[jupton]

I agree, and I would love to even have a comicbookdb branded local database program.

[Dante]

I agree..it would open up things alot..I use Row also and having more than one site (GCD) to get info from would be cool

[tssgery]

Sorry to dredge up an old topic, but I would love an API to pull data from comicbookdb. I would just like the ability to pull information, although being able to submit information would be great as well.

I've considered writing one myself, but am afraid that any page format changes will invalidate all of my work.

Has any more thought gone into delivering something like this?

[mabster]

I agree, and I would love to even have a comicbookdb branded local database program.

I would certainly volunteer Comicster as an official comicbookdb local database program, should such an API ever get implemented.

[coffeemonk]

I certainly have no compunctions about dredging up old topics, especially when they haven't been officially addressed yet...

Has there been any consideration or progress on the API front? The author of ROW put considerable work into gathering data from the site after several requests from his users, only to find that the T&C "prohibits" screen-scraping or other means of data-collection from the site. The prohibition of this is certainly understandable from certain perspectives, but the hope was that the developers of this site would be open to discussion and the possibility of working something out that would be mutually advantageous.

Just think of all the ROW users just sitting on mounds of data that could be easily imported into this site via a "simple" API.

So again, is this even being considered?

[Chris]

I've spoken with both the creators of ROW and Comicster and explained that there are no plans to create an API in the forseeable future. I'm not ruling it out down the line, but it's not a priority right now.

[mabster]

The author of ROW put considerable work into gathering data from the site after several requests from his users, only to find that the T&C "prohibits" screen-scraping or other means of data-collection from the site.

Yeah, that's why I haven't gone down that path with Comicster. I respect that Chris doesn't want people scraping his site.

Something I've thought a bit about is the idea of "live clipboard" (see here). Imagine if each page on comicbookdb.com (and other sites) had the ability to "copy" the data on that page (along with some disclaimer text about where the data came from), and then Comicster (or other programs like RoW) had a "paste" button to insert the copied data into your collection.

Heck - the opposite could hold true too. You could "copy" an issue out of Comicster and then "paste" it into comicbookdb.com.

That way no API would be necessary. It'd be a bit more of a manual process for the user, but it would enable data to seamlessly pass between different sites and/or programs.

Cheers,
Matt

[ProfessorApe]

Hi, I'm new to the forum and registered on the forum to comment on this post.

I've recently started using the CBDB to catalog my comics after trying several Mac applications to do this, all with less than spectacular results. Personally, I'd prefer some kind of desktop app (Mac too please, any decent comic cataloging app is only for Windows). I can see the hesitation to creating an API and letting the world have access to the CBDB info.

However, perhaps something like the Last.FM model would work better? Of course, this would require a software app created by the CBDB team. CBDB could charge for the app, say $25 annually to help offset the cost of development. Those who don't want to buy the app could still use the web interface. An ideal way to do this would be building the app on Adobe's AIR platform (Win/Mac/Linux soon) so the app can be built once instead of maintaining multiple versions.

I think using a desktop app is faster and here's a way to keep a bandwidth load off the CBDB servers: The app could have a local db (sqlite?) that syncs with the CBDB maybe once daily to grab any new titles or other info then sync your collection data back to your account. This way, cataloging could be done offline then just synced back with CBDB once a day (or later) therefore eliminating the need to be online when making ANY changes to your collection.

Thoughts?

[jeffwhitfield]

Hey, ProfessorApe,

I've been on the fence myself regarding the lack of Mac apps for managing a comic collection. As such, it makes total sense to have an online application that is cross-platform.

As the saying goes, "When it doubt, do it yourself". I'm a web developer and work for a small development company. I'm in the process of laying out the foundation for a new online app for managing a comic collection. No word yet on when it will be released. Everything so far is very preliminary.

What you propose is very intesting since it means that CBDB could share in the revenues generated by an online service that uses CBDB data. I'd very much like to have access to the CBDB data in my own app...but it's entirely in the hands of the CBDB admins as to what they want to do with the site.

We can hypothesize and dream up all these cool ideas on what could be done with the CBDB data if there was an API, but until there is one the only solution I have is the create my own database. It would be cool if CBDB was able to allow for two-way synchronization with another database...but, again, the ball in in their court as to how they wish others to use their data.

Jeff

[nariman]

Has the stance on opening the data in the form of an API changed. Having an API that online sites could use to pull in information on any issues would help in promoting the site and establishing it as a source of record. Also being able to download your collection to an desktop application for quick search and view would be awesome.

I have been using the site for a while now and I love it. I think opening the database can really help to take the site to the next level.

[spid]

Has the stance on opening the data in the form of an API changed. Having an API that online sites could use to pull in information on any issues would help in promoting the site and establishing it as a source of record. Also being able to download your collection to an desktop application for quick search and view would be awesome.

I have been using the site for a while now and I love it. I think opening the database can really help to take the site to the next level.

I think I am safe in saying that that stance has not changed.

[Chris]

I think I am safe in saying that that stance has not changed.

Exactly right.

If we change our plans on APIs, rest assured that we'll be trumpeting it far and wide and there won't be any question.

[donnythebowler]

So, I can completely understand that this site is basically ad-driven. We all use it for "free" (although, some time back there was a huge plea for donations... a plea I answered whole-heartedly). Those ads are, I feel, a BIG reason there are no API plans.

For instance, the Intelliscanner company proudly boasts comicbookdb.com integration. Read-only for that matter!?!? The only thing I can guess here is (a) They are scraping or (b) there was a back-room deal where the intelliscanner people are paying for... wait for it... an API!!

Like I said, the site is ad-supported. I have no problem with a 3rd-party company paying for API access. I DO have a problem with posts in this forum saying there is no API. I totally understand that my guesses here may be way off, but (as a web software engineer), I'm not sure how.

I do not want to piss off the creators of this site (who are free to do whatever the hell they want on whatever schedule they want), but I think an API would open this thing up crazy fast.

If you want to charge for developer access to your API, great! Go for it! I'll make an iPhone app and sell it for a dollar to make up for my cost. I think an API is a big deal (in case you haven't noticed).

OK. I'm done. That's all.

[Chris]

As mentioned repeatedly before, there is no API for the site, nor are there plans for one. With the exception of creators and podcasts who we give very limited free access to display things on their website as discussed in other threads in the forum, anyone who says that they integrate with our site is likely illegally scraping it.

I took a cursory look at the Intelliscanner website and didn't find any mention of our site in there. Can you point to where they say that? Considering that we don't track barcodes, I'm not exactly sure what they would be integrating with.

But you can rest assured, there's no need to jump to conclusions as there are no "back-room deals".