An online tool for converting citations to the BibTeX format

  • Published 2005-06-26 (2 years, 10 months ago)

BibConverter is an online tool I have written for converting citations from IEEEXplore and Engineering Village (EV2) to the BibTeX format. It extracts data from text copied directly from the web and outputs BibTeX records that contains much more information than citations exported using the export functionality provided by these bibliographic services. Try it out at http://www.bibconverter.net

Background

BibConverter is the result of laziness and frustration. I wanted to minimize the time from finding an interesting reference online to a new entry in my BibTeX database. Both IEEEXplore and EV2 can export references to plain text, RIS format, and the latter can even export BibTeX records. However, I often find the exported data inadequate for my needs.

I'm currently using JabRef for managing my BibTeX references. It has become a very nice tool, but it's RIS import has some weaknesses. Jabref's RIS import filter is steadily improving, but I was impatient and wanted to try to implement my own RIS to BibTeX converter using Python. I soon realised that writing a good RIS to BibTeX converter is not an easy task. Here's why:

  • A RIS record can store a bibliographic data which has no clear mapping to the BibTeX format
  • A RIS record exported from IEEEXplore is quite different from a record exported from Engineering Village. A filter that can read data from IEEEXplore may fail when reading data from another database.
  • The information exported from online databases is often incomplete

IEEEXplore and especially EV2, stores much information about each reference, including a DOI, author affiliation, conference location and ISSN. All of this information is available when browsing the reference online, but only a subset of this information is exported when using the download functionality. When I discovered this, I gave up my attempt to write a RIS converter. I wanted to have access to all available information.

After some thinking I got an idea. Why not extract information directly from the web-page using copy and paste? The online record views at IEEEXplore and EV2 have a well defined structure. With some clever text processing it's relatively easy to extract all the available information. The extra amount of information makes it easier to output correct and complete BibTeX records. Add a web interface, and you'll have a very useful tool.

Future plans

I plan to add more features to BibConverter as I and other users need them. Don't hesitate to contact me if you find bugs or have a feature request. Eventually I'll make the source code available for download.

Comments

  • #1 Richard K., May 6, 2007 at 6:19 p.m.

    Hello, thanks for your nice tool, I'm just beginning to dive into the scientific publication field- and your bibtex converter saved me a lot of time. Here is another tool which does the springer-link conversion. http://www.cs.usyd.edu.au/~niu/cgi-bin/springer.cgi It's also written in python- maybe you could join forces with the author of this tool.

    Greetings, Richard

  • #2 Kjell Magne Fauske, May 6, 2007 at 8:49 p.m.

    Thanks Richard, for the positive feedback and the link to the Springer converter. I was not aware of that one.

    The converter uses a different screen scraping approach than my BibConverter. It parses the HTML code, while BibConverter uses the clipboard. Interesting. It is probably easier to extract the data that way. My approach requires carefully crafted regular expressions that sometimes fail in unexpected ways.

  • #3 Luciana, November 6, 2007 at 11:30 a.m.

    I love your tool thanks for making it public!!!

  • #4 Alfredo, December 8, 2007 at 11:59 p.m.

    Thank you Kjell, it is a great tool. I am working on my thesis and lately I have to deal with citations from WoS, Nature, Science and PNAS which do not output it bibliography in bibtex format, I am sure this is a commercial issue of WoS and its product EndNote, and some obscure shameful deal with those journals.

    I am big fan of your site by the way.

  • #5 Kjell Magne Fauske, December 9, 2007 at 7:45 p.m.

    Thank you Alfredo for you comment! Nice to know that people use the WoS converter. Compared to the IEEEXplore converter, it receives very little traffic.

  • #6 T3, February 11, 2008 at 3:19 p.m.

    Kjell - this tool is a real time saver! Simply fantastic! I mostly use WoS converter, since that gives access to the huge ISI database. Until recently, that is, because ISI revamped their WoS portal and converter doesn't work with it anymore :( I hope you will find some time to update your tool. The required changes seem minor - from what I could figure out some fields are not on separate lines anymore and title is not marked with "Title: ". Making those changes seemed to fix the problem.

    Best wishes, T3

  • #7 Kjell Magne Fauske, February 12, 2008 at 9:12 p.m.

    @T3 Thank you for reporting the issue with the new WoS portal. I have made some updates to BibConverter and the WoS converter is now working again (I hope). Please report any issues. I have not tested the new version thoroughly yet.

  • #8 T3, February 13, 2008 at 12:17 a.m.

    I've just had a quick look. It does detect correctly bibliographic fields now :) except that title has no markup in this new version of WoS - it must be manually preceded with "title:" otherwise I get "Error: Could not extract any data". But that's only 1 or 2 sec. more to get citation into JabRef so it's no problem.

    Thanks again for this great tool.

  • #9 Kjell Magne Fauske, February 13, 2008 at 8:19 a.m.

    @T3 Could you send me the text you are trying to convert? I can not reproduce the error. Please use the contact form because it does not modify the text.

    A few tips:

    • Select and copy all the text on the page (Ctrl+A). That makes it easier for the converter to locate the correct information.
    • You can also use the 'Output Record' feature. Select the 'Full record' option and press the 'Print' button. You can also convert multiple records this way.
  • #10 Kjell Magne Fauske, February 13, 2008 at 1:36 p.m.

    @T3 The problems seems to be that output varies slightly between browsers like Internet Explorer and Firefox. I have updated BibConverter to reflect this. Hopefully it is working for you now. I have tested it in IE, Firefox and Opera.

  • #11 T3, February 13, 2008 at 2:26 p.m.

    It didn't occur to me that it might depend on the browser used but you are absolutely right. I mostly use Opera (v9.20) and it works there great now.

    Thanks again!

  • #12 Kjell Magne Fauske, February 13, 2008 at 2:33 p.m.

    I'm glad to hear that it is working again. This shows how important it is to test on different platforms. I tend to forget that sometimes.

    One strange thing when using Opera (9.0), is that I can't find the 'Full record' option for output.

  • #13 Kjell Magne Fauske, February 14, 2008 at 10 a.m.

    The WoS converter has been updated again. All reported bugs have been fixed (I hope). There are probably still special cases that are not handled correctly. If you report them I will do my best to fix them.

Post a comment

Markdown syntax enabled

(required)

(required, but will not be published)