ODF file formats vs Zip

Asked by 1 year ago
When working with big files, in my case spreadsheets, but possibly other types of office files, saving the file will in some cases take a lot of time. This is particularly annoying when auto-saving is enabled. As I understand it, an ODF is a couple of files, most of them XML files, brought together in a single file, then compressed to the zip format. Does the ODF standard specify the compression ratio? If not, it would be convenient if the user could specify that. For example, if I prefer saving to be as fast as possible, I could specify no compression at all, just bring the files together in a tar-ball (if that's allowed) or as an uncompressed zip. I don't know how much of the required time to save a file is used for compression, but I imagine that there is room for speed enhancements here. If this is not the way to go, maybe the extension could change as well, indicating this is another file format, although conversion to and from ODF should be very straight forward… Thoughts about this? Personally I just thing that something must be done about the auto-save speed. And also, when opening a spreadsheet, ”adapting row heights”, what is that? Is that really necessary? Shouldn't row heights already be specified in the ODF file? It's maybe not the same subject, but in a way it is about time consuming saving and opening of different kinds of ODF files… Johnny Rosenberg

Your Answer

Name:
Reply:

All Answers

Answer by 1 year ago
Hi Johnny, Johnny Rosenberg schrieb: There are two methods possible STORED and DEFLATED, see http://docs.oasis-open.org/office/v1.2/os/OpenDocument-v1.2-os-part3.html, section 2.2. If not, it would Using another compression is still .zip file format. ODF has a flat file format without container too. This is implemented in LO but not in AOO. But in the flat format all pictures are stored in base64, because there is no folder to store them in original format. It would need tests to see, whether the method STORED is significant faster. Kind regards Regina
Answer by 1 year ago
Regina is correct about the only two compressions. As far as I know, there is no way to control which compression is used. (If you save with Password, all files are always compressed.) Most of the time DEFLATE is used (although there are two files that are not usually compressed, apparently to make metadata mining simpler for non-encrypted packages). There is currently no way to control the compression in AOO. (The ODF specification simply stipulates the compression that must be used when compression is done, not whether compression is done for parts of unencrypted packages.) I don't think it is the compression that is responsible for the slow-downs, it has to do with other work that goes on in order to save a file. If you are careful about regularly saving manually while you are working, and you work into a new copy so the starting version can't be damaged, you can disable auto-save to avoid being interrupted in the midst of something you are doing. There may be some glitches that cause the time to increase in certain situations and those are caught from time to time. Using the latest version usually includes those improvements. I suspect there are some other performance issues around Save (and Auto-Save) that are more involved. - Dennis -----Original Message----- From: Regina Henschel [mailto:rb.henschel [ at ] t-online.de] Sent: Sunday, June 9, 2013 11:41 AM To: users [ at ] openoffice.apache.org Subject: Re: [DISCUSS] ODF file formats vs Zip Hi Johnny, Johnny Rosenberg schrieb: There are two methods possible STORED and DEFLATED, see http://docs.oasis-open.org/office/v1.2/os/OpenDocument-v1.2-os-part3.html, section 2.2. If not, it would Using another compression is still .zip file format. ODF has a flat file format without container too. This is implemented in LO but not in AOO. But in the flat format all pictures are stored in base64, because there is no folder to store them in original format. It would need tests to see, whether the method STORED is significant faster. Kind regards Regina
Answer by 1 year ago
I always have auto-save off for those reasons, and instead I'm in the habit of very frequently saving the file (which takes about half a second using Ctrl+S (on Windows, may be different on other operating systems). That way, the save can happen when I want it to (like when I'm looking at what I've written or thinking about something ...). Also, I don't know if this is the case with auto-save, but when manually saving, I'm pretty sure that things can not be "undone" prior to the save. If this is indeed the case with auto-save, this is another very good reason not to use it. You may have done something you really want to undo, but if the auto-save happens, you can't. (Again, I'm not positive whether this is the case with auto-save like with manual save, but I'm guessing it may be.)
Answer by 1 year ago
To reduce the save time, split the files into many, and have links between files. This Feature should be available in AOO. Then your saving time will be much faster. In case of file corruption due to any reason, only that file get corrupted and the remaining file may remain safe With Warm Regards V.Kadal Amutham 919444360480 914422396480
Answer by 1 year ago
2013/6/10 Richard Detwiler RLShadow [ at ] aol.com: I think the time will be more dependent on the contents of your document and the speed of your hardware. When I say several seconds, I am talking about a spreadsheet with thousands of rows and maybe 20-30 columns and 5-10 sheets (not all of them that populated, though). Of course, when working with a smaller document, saving time is not an issue at all.
Answer by 1 year ago
Quoted message by Richard Detwiler 1 year ago
I always have auto-save off for those reasons, and instead I'm in the habit of very frequently saving the file (which takes about half a second using Ctrl+S (on Windows, may be different on other operating systems). That way, the save can happen when I want it to (like when I'm looking at what I've written or thinking about something ...). Also, I don't know if this is the case with auto-save, but when manually saving, I'm pretty sure that things can not be "undone" prior to the save. If this is indeed the case with auto-save, this is another very good reason not to use it. You may have done something you really want to undo, but if the auto-save happens, you can't. (Again, I'm not positive whether this is the case with auto-save like with manual save, but I'm guessing it may be.)
2013/6/10 Kadal Amutham vkadal [ at ] gmail.com: It would probably help some, but I would still end up with at least one giant file which holds all the main data of thousands of rows (increasing all the time) and maybe 10-20 columns. Still, finding a way to make saving faster wouldn't kill someone, would it? It seems like every time someone suggest an improvement, or at least a change, there are numerous arguments why this shouldn't be done, no matter what it is. Maybe ”development” doesn't mean the same thing to everyone. So well, let's just forget about all this and continue our lives. Sorry for annoying everyone. Johnny Rosenberg
Answer by 1 year ago
From: Richard Detwiler RLShadow [ at ] aol.com To: users [ at ] openoffice.apache.org Date: Sun, 09 Jun 2013 20:16:32 -0400
Answer by 1 year ago
At my age, I can't be trusted to remember to save it with enough frequency to be beneficial, so I'd rather "suffer" the few moments' delay rather than lose several hours of work which has happened to me in other apps that don't have such a feature. Dale Erwin Jr. 28 de Julio 657, Depto. 03 Magdalena del Mar, Lima 17 PERU http://leather.casaerwin.org
Answer by 1 year ago
Quoted message by Regina Henschel 1 year ago
Hi Johnny, Johnny Rosenberg schrieb: There are two methods possible STORED and DEFLATED, see http://docs.oasis-open.org/office/v1.2/os/OpenDocument-v1.2-os-part3.html, section 2.2. If not, it would Using another compression is still .zip file format. ODF has a flat file format without container too. This is implemented in LO but not in AOO. But in the flat format all pictures are stored in base64, because there is no folder to store them in original format. It would need tests to see, whether the method STORED is significant faster. Kind regards Regina
On Sun, Jun 9, 2013 at 4:01 PM, Dennis E. Hamilton Does anyone know whether AOO is smart enough to not waste time trying to compress already compressed files, like PNG images? This could make a big difference in presentations. -Rob
Answer by 1 year ago
Quoted message by Richard Detwiler 1 year ago
I always have auto-save off for those reasons, and instead I'm in the habit of very frequently saving the file (which takes about half a second using Ctrl+S (on Windows, may be different on other operating systems). That way, the save can happen when I want it to (like when I'm looking at what I've written or thinking about something ...). Also, I don't know if this is the case with auto-save, but when manually saving, I'm pretty sure that things can not be "undone" prior to the save. If this is indeed the case with auto-save, this is another very good reason not to use it. You may have done something you really want to undo, but if the auto-save happens, you can't. (Again, I'm not positive whether this is the case with auto-save like with manual save, but I'm guessing it may be.)
Have you seen http://wiki.openoffice.org/wiki/Performance? There are several issues about performance Regards Oliver GnuPG key 0xCFD04A45: 8822 057F 4956 46D3 352C 1A06 4E2C AB40 CFD0 4A45
Answer by 1 year ago
Quoted message by Dennis E. Hamilton 1 year ago
Regina is correct about the only two compressions. As far as I know, there is no way to control which compression is used. (If you save with Password, all files are always compressed.) Most of the time DEFLATE is used (although there are two files that are not usually compressed, apparently to make metadata mining simpler for non-encrypted packages). There is currently no way to control the compression in AOO. (The ODF specification simply stipulates the compression that must be used when compression is done, not whether compression is done for parts of unencrypted packages.) I don't think it is the compression that is responsible for the slow-downs, it has to do with other work that goes on in order to save a file. If you are careful about regularly saving manually while you are working, and you work into a new copy so the starting version can't be damaged, you can disable auto-save to avoid being interrupted in the midst of something you are doing. There may be some glitches that cause the time to increase in certain situations and those are caught from time to time. Using the latest version usually includes those improvements. I suspect there are some other performance issues around Save (and Auto-Save) that are more involved. - Dennis -----Original Message----- From: Regina Henschel [mailto:rb.henschel [ at ] t-online.de] Sent: Sunday, June 9, 2013 11:41 AM To: users [ at ] openoffice.apache.org Subject: Re: [DISCUSS] ODF file formats vs Zip Hi Johnny, Johnny Rosenberg schrieb: There are two methods possible STORED and DEFLATED, see http://docs.oasis-open.org/office/v1.2/os/OpenDocument-v1.2-os-part3.html, section 2.2. If not, it would Using another compression is still .zip file format. ODF has a flat file format without container too. This is implemented in LO but not in AOO. But in the flat format all pictures are stored in base64, because there is no folder to store them in original format. It would need tests to see, whether the method STORED is significant faster. Kind regards Regina
Hi Rob, AFAIK Images are not compressed; i stumbled over this with the added SVG format which is still added uncompressed (we have a task for it). HTH! Sincerely, Armin
Answer by 1 year ago
Quoted message by Dennis E. Hamilton 1 year ago
Regina is correct about the only two compressions. As far as I know, there is no way to control which compression is used. (If you save with Password, all files are always compressed.) Most of the time DEFLATE is used (although there are two files that are not usually compressed, apparently to make metadata mining simpler for non-encrypted packages). There is currently no way to control the compression in AOO. (The ODF specification simply stipulates the compression that must be used when compression is done, not whether compression is done for parts of unencrypted packages.) I don't think it is the compression that is responsible for the slow-downs, it has to do with other work that goes on in order to save a file. If you are careful about regularly saving manually while you are working, and you work into a new copy so the starting version can't be damaged, you can disable auto-save to avoid being interrupted in the midst of something you are doing. There may be some glitches that cause the time to increase in certain situations and those are caught from time to time. Using the latest version usually includes those improvements. I suspect there are some other performance issues around Save (and Auto-Save) that are more involved. - Dennis -----Original Message----- From: Regina Henschel [mailto:rb.henschel [ at ] t-online.de] Sent: Sunday, June 9, 2013 11:41 AM To: users [ at ] openoffice.apache.org Subject: Re: [DISCUSS] ODF file formats vs Zip Hi Johnny, Johnny Rosenberg schrieb: There are two methods possible STORED and DEFLATED, see http://docs.oasis-open.org/office/v1.2/os/OpenDocument-v1.2-os-part3.html, section 2.2. If not, it would Using another compression is still .zip file format. ODF has a flat file format without container too. This is implemented in LO but not in AOO. But in the flat format all pictures are stored in base64, because there is no folder to store them in original format. It would need tests to see, whether the method STORED is significant faster. Kind regards Regina
Looking into a document having images with WinZip, it appears that GIF and PNG files are not compressed and SVM files are (with great improvement). The content.xml files, which can be megabytes long, benefit greatly from compression (9:1 easily). Apache OpenOffice 3.4.1 and older versions of OpenOffice.org will compress the Thumbnail PNG. Not sure why, but it is a small file so it shouldn't matter in terms of Save performance. - Dennis PS: I don't know whether uncompressed results are also obtained by attempting compression and reverting to STORED when the compression is unsuccessful. Some software does that sort of thing. (I have a recollection that DEFLATE can also produce uncompressed sections on discovery of their uncompressability, but the result won't be the same size as the original. I don't know if the DEFLATE compression used will produce those.) -----Original Message----- From: Rob Weir [mailto:robweir [ at ] apache.org] Sent: Monday, June 10, 2013 05:45 AM To: users [ at ] openoffice.apache.org; Dennis Hamilton Subject: Re: [DISCUSS] ODF file formats vs Zip On Sun, Jun 9, 2013 at 4:01 PM, Dennis E. Hamilton Does anyone know whether AOO is smart enough to not waste time trying to compress already compressed files, like PNG images? This could make a big difference in presentations. -Rob [ ... ]
Answer by 1 year ago
Quoted message by Rob Weir 1 year ago
On Sun, Jun 9, 2013 at 4:01 PM, Dennis E. Hamilton Does anyone know whether AOO is smart enough to not waste time trying to compress already compressed files, like PNG images? This could make a big difference in presentations. -Rob
Hi, i prefer *linking* pictures instead of saving them every time - and especially for bigger writer documents, use the "Working with Master Documents and Subdocuments" feature. Regards Oliver
Answer by 1 year ago
Quoted message by Rob Weir 1 year ago
On Sun, Jun 9, 2013 at 4:01 PM, Dennis E. Hamilton Does anyone know whether AOO is smart enough to not waste time trying to compress already compressed files, like PNG images? This could make a big difference in presentations. -Rob
On Mon, Jun 10, 2013 at 12:22 PM, Dennis E. Hamilton If you really want to try a monster test case, try the spreadsheets from this old ZDNet article from 2005: http://www.zdnet.com/blog/ou/performance-analysis-of-openoffice-and-ms-office/120 They are in pre-ODF XML formats, but can easily be converted. Try it as DOC and as ODS. The files themselves look quite reasonable, due to the ZIP compression. But then try unzipping the file. You'll see the content.xml is much, much larger. The problem we have with large ODF spreadsheets is our cell-by-cell table markup is very verbose. We also lack a "string-pool" structure in the markup to deal with repeated strings, which are common in database-like uses of a spreadsheet. Regards, -Rob
Answer by 1 year ago
Quoted message by Dennis E. Hamilton 1 year ago
Regina is correct about the only two compressions. As far as I know, there is no way to control which compression is used. (If you save with Password, all files are always compressed.) Most of the time DEFLATE is used (although there are two files that are not usually compressed, apparently to make metadata mining simpler for non-encrypted packages). There is currently no way to control the compression in AOO. (The ODF specification simply stipulates the compression that must be used when compression is done, not whether compression is done for parts of unencrypted packages.) I don't think it is the compression that is responsible for the slow-downs, it has to do with other work that goes on in order to save a file. If you are careful about regularly saving manually while you are working, and you work into a new copy so the starting version can't be damaged, you can disable auto-save to avoid being interrupted in the midst of something you are doing. There may be some glitches that cause the time to increase in certain situations and those are caught from time to time. Using the latest version usually includes those improvements. I suspect there are some other performance issues around Save (and Auto-Save) that are more involved. - Dennis -----Original Message----- From: Regina Henschel [mailto:rb.henschel [ at ] t-online.de] Sent: Sunday, June 9, 2013 11:41 AM To: users [ at ] openoffice.apache.org Subject: Re: [DISCUSS] ODF file formats vs Zip Hi Johnny, Johnny Rosenberg schrieb: There are two methods possible STORED and DEFLATED, see http://docs.oasis-open.org/office/v1.2/os/OpenDocument-v1.2-os-part3.html, section 2.2. If not, it would Using another compression is still .zip file format. ODF has a flat file format without container too. This is implemented in LO but not in AOO. But in the flat format all pictures are stored in base64, because there is no folder to store them in original format. It would need tests to see, whether the method STORED is significant faster. Kind regards Regina
Could anyone explain why it is necessary to compress Autorecovery information. it is only used to recover when the system crashes. In my opinion it should only compress when a user close the document.
Answer by 1 year ago
Quoted message by Richard Detwiler 1 year ago
I always have auto-save off for those reasons, and instead I'm in the habit of very frequently saving the file (which takes about half a second using Ctrl+S (on Windows, may be different on other operating systems). That way, the save can happen when I want it to (like when I'm looking at what I've written or thinking about something ...). Also, I don't know if this is the case with auto-save, but when manually saving, I'm pretty sure that things can not be "undone" prior to the save. If this is indeed the case with auto-save, this is another very good reason not to use it. You may have done something you really want to undo, but if the auto-save happens, you can't. (Again, I'm not positive whether this is the case with auto-save like with manual save, but I'm guessing it may be.)
Kadal Amutham (2013-06-10 03:40): Would be extremely helpful if OO would allow creating a master document that would automatically link chapters (allowing going back and forth, adding new one, adding existing file as a chapter...). Then you would have to be able to easily generate common TOC and merge documents into single PDF and such things. This would make the scenario feasible. Otherwise it's kind of in practical to divide documents. Regards, Nux.
Answer by 1 year ago
Quoted message by Kadal Amutham 1 year ago
To reduce the save time, split the files into many, and have links between files. This Feature should be available in AOO. Then your saving time will be much faster. In case of file corruption due to any reason, only that file get corrupted and the remaining file may remain safe With Warm Regards V.Kadal Amutham 919444360480 914422396480
It does. I use it for all my books even tough the tutorial discourages it. Dale Erwin Jr. 28 de Julio 657, Depto. 03 Magdalena del Mar, Lima 17 PERU http://leather.casaerwin.org
Answer by 1 year ago
Le 10/06/2013 02:16, Richard Detwiler a ?crit : No. The auto-save operation does not reset the undo history (same with standard save). Hagar
Answer by 1 year ago
Quoted message by Richard Detwiler 1 year ago
I always have auto-save off for those reasons, and instead I'm in the habit of very frequently saving the file (which takes about half a second using Ctrl+S (on Windows, may be different on other operating systems). That way, the save can happen when I want it to (like when I'm looking at what I've written or thinking about something ...). Also, I don't know if this is the case with auto-save, but when manually saving, I'm pretty sure that things can not be "undone" prior to the save. If this is indeed the case with auto-save, this is another very good reason not to use it. You may have done something you really want to undo, but if the auto-save happens, you can't. (Again, I'm not positive whether this is the case with auto-save like with manual save, but I'm guessing it may be.)
Well, the Saved copy doesn't preserve Undo history, and Hagar is correct: Performing Save does not clear the Undo history for the document that remains open. Whether an AutoSave recovery will recover the Undo history to that point is something that can be verified by an experiment of some kind. I also realize that I don't know if AutoSave is completely separate from the document recovery material used after crashes and other situations or that's what AutoSave is updating. More experimentation! Helpful documentation? - Dennis -----Original Message----- From: Hagar Delest [mailto:hagar.delest [ at ] laposte.net] Sent: Sunday, June 23, 2013 03:37 AM To: users [ at ] openoffice.apache.org Subject: Re: [DISCUSS] ODF file formats vs Zip Le 10/06/2013 02:16, Richard Detwiler a écrit : No. The auto-save operation does not reset the undo history (same with standard save). Hagar
Answer by 1 year ago
Quoted message by Richard Detwiler 1 year ago
I always have auto-save off for those reasons, and instead I'm in the habit of very frequently saving the file (which takes about half a second using Ctrl+S (on Windows, may be different on other operating systems). That way, the save can happen when I want it to (like when I'm looking at what I've written or thinking about something ...). Also, I don't know if this is the case with auto-save, but when manually saving, I'm pretty sure that things can not be "undone" prior to the save. If this is indeed the case with auto-save, this is another very good reason not to use it. You may have done something you really want to undo, but if the auto-save happens, you can't. (Again, I'm not positive whether this is the case with auto-save like with manual save, but I'm guessing it may be.)
I don't think that's true. I think that if you close a document without saving, it will revert back to its state before opening it even if it has been autosaved during the editing process. Only if you explicitly save it will you be unable to do that. It's as though the only purpose of autosave is the event of a system failure. In the even of a system failure you have the option at reboot of recuperating all changes made up to the last autosave, but you can decline. Dale Erwin Jr. 28 de Julio 657, Depto. 03 Magdalena del Mar, Lima 17 PERU http://leather.casaerwin.org
Answer by 1 year ago
Quoted message by Richard Detwiler 1 year ago
I always have auto-save off for those reasons, and instead I'm in the habit of very frequently saving the file (which takes about half a second using Ctrl+S (on Windows, may be different on other operating systems). That way, the save can happen when I want it to (like when I'm looking at what I've written or thinking about something ...). Also, I don't know if this is the case with auto-save, but when manually saving, I'm pretty sure that things can not be "undone" prior to the save. If this is indeed the case with auto-save, this is another very good reason not to use it. You may have done something you really want to undo, but if the auto-save happens, you can't. (Again, I'm not positive whether this is the case with auto-save like with manual save, but I'm guessing it may be.)
Are you saying that if, while editing a document, I do an explicit save, I can still use the undo feature for anything I have changed during the current session? Dale Erwin Jr. 28 de Julio 657, Depto. 03 Magdalena del Mar, Lima 17 PERU http://leather.casaerwin.org
Answer by 1 year ago
Quoted message by Hagar Delest 1 year ago
Le 10/06/2013 02:16, Richard Detwiler a ?crit : No. The auto-save operation does not reset the undo history (same with standard save). Hagar
Yes. (Up to the maximum number of Undos, of course - is that 30 by default?) What happened when you tested this yourself? Brian Barker
Answer by 1 year ago
Quoted message by Dale Erwin 1 year ago
Are you saying that if, while editing a document, I do an explicit save, I can still use the undo feature for anything I have changed during the current session? Dale Erwin Jr. 28 de Julio 657, Depto. 03 Magdalena del Mar, Lima 17 PERU http://leather.casaerwin.org
I haven't actually tested this explicitly yet, but I certainly will do so very soon. I just never thought it would be possible so I never tried it. I do know, however, that no many how many times the file has been autosaved, if I close the document without an explicit save, it will revert back to the same state it was in before it was opened. Dale Erwin Jr. 28 de Julio 657, Depto. 03 Magdalena del Mar, Lima 17 PERU http://leather.casaerwin.org
Answer by 1 year ago
Quoted message by Brian Barker 1 year ago
Yes. (Up to the maximum number of Undos, of course - is that 30 by default?) What happened when you tested this yourself? Brian Barker
Good. If this were not so, surely no-one would dream of using the product? Brian Barker
Answer by 12 months ago
Quoted message by Hagar Delest 1 year ago
Le 10/06/2013 02:16, Richard Detwiler a ?crit : No. The auto-save operation does not reset the undo history (same with standard save). Hagar
Of course, if you close your document without saving, it will leave your document in the state you last saved it manually. AutoSave is only used in case of crash. Some users think that it's feature that make regular backups, actually saving the file for future use, which is not true. Perhaps the name should be changed to "Crash info"? Too negative perhaps. Hagar Le 24/06/2013 01:10, Dale Erwin a ?crit :
Answer by 12 months ago
Quoted message by Hagar Delest 1 year ago
Le 10/06/2013 02:16, Richard Detwiler a ?crit : No. The auto-save operation does not reset the undo history (same with standard save). Hagar
Le 23/06/2013 18:14, Dennis E. Hamilton a ?crit : No, AutoSave does not recover the undo history. AutoSave is only used in case of crash for the recovery process. Hagar