Thursday, July 12, 2012

Improved ODS Load Times

Edited to include links to the test files that were used.

Under the guidance of Kohei Yoshida and Markus Mohrhard, I have been working to shorten the time it takes to open an ODS file.  These shorter load times may be more noticeable depending on the content and size of the file that is being loaded.  Although this work should result in all ODS files having at least a small improvement in load times, let's take a look at a couple of extreme cases.

Please keep in mind these are not rigorously, scientifically performed tests.  They are just to give an idea of the improvements we are making.  I did these tests on a machine with a 3.2GHz AMD Athon 64 X2 Dual Core Processor 6400+ and 8GB of RAM running 64-bit GNU/Linux.  I used LibreOffice 3.5.4 and the latest build of my feature branch which gradually gets merged to master (LibreOffice 3.7).

Test 1 used an ODS file with a single sheet containing 10,001 rows and 149 columns of numbers and simple formulas.  LibreOffice 3.5.4 takes 20 seconds to load this file, and LibreOffice 3.7 takes 16 seconds.  That's four seconds quicker which is a 20% reduction in load time.  Ok, that's not bad.

Test 1


Test 2 used an ODS file with a single sheet containing 5,230 rows and 189 columns of matrix cells with complex formulas.  LibreOffice 3.5.4 takes 26 seconds to load this file, and LibreOffice 3.7 takes 13 seconds.  That's 13 seconds quicker which is a 50% reduction in load time!

Test 2


Also, take a look at Kohei's post to see some other awesome ODS load time improvements made last year.  (I attempted to mimic his format for the load time charts and drew inspiration for my whole blog post from his post.)

So we've made progress, but we are all continuing to work to improve LibreOffice for a better user experience.

14 comments:

  1. I updated the post to include links to the test files I used.

    ReplyDelete
  2. More work was done later on the matrix import to account for special cases of formulas that always need to be recalculated. At first, we lost some performance having to account for these cases. Fortunately, the performance reported in this post has now been regained. (Although, it has not yet been merged to master.)

    ReplyDelete
    Replies
    1. Hello Daniel,

      I would like to bring your attention to this bug report:

      https://www.libreoffice.org/bugzilla/show_bug.cgi?id=54638

      I can confirm that load and save time of ODS files have dramatically increased from version 3.4 to 3.6.

      Same PC, same ODS: load time went up from 11.5/12.5s (LibO3.4/3.5) to 30s (LibO3.6)
      save time went up from 6.5/7.5s (LibO3.4/3.5) to 30s (LibO3.6).

      Can you please run your test against the documents attached to that bug report and make sure the next LibO release fixes this issue?

      Also, I find it interesting that MS Excel can open an ODS file much faster than LibO itself: (3.7s vs 11.8s)

      http://ask.libreoffice.org/question/154/why-in-calc-saving-is-so-slow-os-win7x64/

      Delete
    2. Hi, sorry, it took so long for me to reply. I'm just now getting back into LibreOffice development. It looks like there have been some updates to that bug report, and it looks like this will be one of the performance issues we will address in the future.

      Delete
  3. Thank you for the work!
    I have sheets with cca 140000 rows and it take cca 10 minuts to load/save. Nearly impossible to use. But now I can see some progress here. Great!

    ReplyDelete
  4. I'm be happy to see that there is a quicker LO.
    But i see that there is by the newer LO 3.6.0 a step backwards from 3.5.4 with the calculation time by calc. See in bugzilla issue 57028, i hope that this issue is solve for LO 4.0!

    ReplyDelete
    Replies
    1. Hi, sorry it took so long to reply. It looks like https://www.libreoffice.org/bugzilla/show_bug.cgi?id=57028 has been fixed by Markus (http://cgit.freedesktop.org/libreoffice/core/commit/?id=7c528f40e0aea9386d175b007974c02d265f96c6).

      Delete
  5. This is Amazing, Daniel !
    thank you for your contribution to the world.

    ReplyDelete