[Bug 152336] Improve CSV field separator detection/behavior when several separators are selected on import
https://bugs.documentfoundation.org/show_bug.cgi?id=152336 Eike Rathke changed: What|Removed |Added See Also||https://bugs.documentfounda ||tion.org/show_bug.cgi?id=15 ||8025 -- You are receiving this mail because: You are the assignee for the bug.
[Bug 152336] Improve CSV field separator detection/behavior when several separators are selected on import
https://bugs.documentfoundation.org/show_bug.cgi?id=152336 Mike Kaganski changed: What|Removed |Added See Also||https://bugs.documentfounda ||tion.org/show_bug.cgi?id=15 ||3457 -- You are receiving this mail because: You are the assignee for the bug.
[Bug 152336] Improve CSV field separator detection/behavior when several separators are selected on import
https://bugs.documentfoundation.org/show_bug.cgi?id=152336 Mike Kaganski changed: What|Removed |Added CC||b...@bertenselena.net --- Comment #20 from Mike Kaganski --- *** Bug 160742 has been marked as a duplicate of this bug. *** -- You are receiving this mail because: You are the assignee for the bug.
[Bug 152336] Improve CSV field separator detection/behavior when several separators are selected on import
https://bugs.documentfoundation.org/show_bug.cgi?id=152336 Mike Kaganski changed: What|Removed |Added Attachment #193599|application/vnd.ms-excel|text/csv mime type|| -- You are receiving this mail because: You are the assignee for the bug.
[Bug 152336] Improve CSV field separator detection/behavior when several separators are selected on import
https://bugs.documentfoundation.org/show_bug.cgi?id=152336 --- Comment #19 from Laurent Balland --- (In reply to Laurent Balland from comment #18) > For next times, change only these options: > - Character set=Western Europe (ISO-8859-15/EURO) > That's all :-) Ooops! It was missing > - select 2nd column and change Column type to Date (DMY) But Other options, like "Detect special numbers" or "Evaluate formulas", where preselected or not according to the last use of the dialog -- You are receiving this mail because: You are the assignee for the bug.
[Bug 152336] Improve CSV field separator detection/behavior when several separators are selected on import
https://bugs.documentfoundation.org/show_bug.cgi?id=152336 Laurent Balland changed: What|Removed |Added Attachment #193599|text/plain |application/vnd.ms-excel mime type|| -- You are receiving this mail because: You are the assignee for the bug.
[Bug 152336] Improve CSV field separator detection/behavior when several separators are selected on import
https://bugs.documentfoundation.org/show_bug.cgi?id=152336 Laurent Balland changed: What|Removed |Added Attachment #193599|application/vnd.ms-excel|text/plain mime type|| -- You are receiving this mail because: You are the assignee for the bug.
[Bug 152336] Improve CSV field separator detection/behavior when several separators are selected on import
https://bugs.documentfoundation.org/show_bug.cgi?id=152336 --- Comment #18 from Laurent Balland --- Created attachment 193599 --> https://bugs.documentfoundation.org/attachment.cgi?id=193599=edit CSV file with separators: no comma, single quote, ° character Before the change, when opening this CSV file for 1st time, change options: - Character set=Western Europe (ISO-8859-15/EURO) - Locale=French (France) - Separated by: uncheck comma, check Other and insert '° in Other field - check Detect special numbers - select 2nd column and change Column type to Date (DMY) For next times, change only these options: - Character set=Western Europe (ISO-8859-15/EURO) That's all :-) With the change, you need to change all options like the 1st time :-( -- You are receiving this mail because: You are the assignee for the bug.
[Bug 152336] Improve CSV field separator detection/behavior when several separators are selected on import
https://bugs.documentfoundation.org/show_bug.cgi?id=152336 Laurent Balland changed: What|Removed |Added CC||jumbo4...@yahoo.fr --- Comment #17 from Laurent Balland --- There is a drawback with this enhancement: separators options are no more saved. Before this enhancement, when opening a CSV with some weird separators, you will retrieve your last choice of separators. Then, if you open several files with the same pattern, you will have the right default preselection of separators after the first opening. With this enhancement, the separators options are always reset to default. I'm going to add an example. -- You are receiving this mail because: You are the assignee for the bug.
[Bug 152336] Improve CSV field separator detection/behavior when several separators are selected on import
https://bugs.documentfoundation.org/show_bug.cgi?id=152336 --- Comment #16 from Gabriel Masei --- (In reply to Gabriel Masei from comment #15) > (In reply to Mike Kaganski from comment #13) > > (In reply to m_a_riosv from comment #12) > > > > This is a very important issue. Please file it as a regression bug. > > Gabriel: please consider making this an opt-in (an option in filter option; > > a "detect me" kind of a button in the dialog...) What do you think? > > I provided a quick fix only for locale issue here: > https://gerrit.libreoffice.org/c/core/+/165905 > > I'll explore the possibility to preserve the compatibility with saved > settings. This needs a bit of time to find a good solution to mitigate the > conflict between the two features. In fact it should work for all options except "Character set" and "Separator Options" -- You are receiving this mail because: You are the assignee for the bug.
[Bug 152336] Improve CSV field separator detection/behavior when several separators are selected on import
https://bugs.documentfoundation.org/show_bug.cgi?id=152336 --- Comment #15 from Gabriel Masei --- (In reply to Mike Kaganski from comment #13) > (In reply to m_a_riosv from comment #12) > > This is a very important issue. Please file it as a regression bug. > Gabriel: please consider making this an opt-in (an option in filter option; > a "detect me" kind of a button in the dialog...) What do you think? I provided a quick fix only for locale issue here: https://gerrit.libreoffice.org/c/core/+/165905 I'll explore the possibility to preserve the compatibility with saved settings. This needs a bit of time to find a good solution to mitigate the conflict between the two features. -- You are receiving this mail because: You are the assignee for the bug.
[Bug 152336] Improve CSV field separator detection/behavior when several separators are selected on import
https://bugs.documentfoundation.org/show_bug.cgi?id=152336 --- Comment #14 from ady --- (In reply to Mike Kaganski from comment #13) > (In reply to m_a_riosv from comment #12) > > This is a very important issue. Please file it as a regression bug. That is now tdf#160582. -- You are receiving this mail because: You are the assignee for the bug.
[Bug 152336] Improve CSV field separator detection/behavior when several separators are selected on import
https://bugs.documentfoundation.org/show_bug.cgi?id=152336 ady changed: What|Removed |Added See Also||https://bugs.documentfounda ||tion.org/show_bug.cgi?id=16 ||0582 -- You are receiving this mail because: You are the assignee for the bug.
[Bug 152336] Improve CSV field separator detection/behavior when several separators are selected on import
https://bugs.documentfoundation.org/show_bug.cgi?id=152336 m_a_riosv changed: What|Removed |Added Blocks||160582 Referenced Bugs: https://bugs.documentfoundation.org/show_bug.cgi?id=160582 [Bug 160582] FILEOPEN: CSV, locale setting is not preserved in the import window (regression tdf#152336) -- You are receiving this mail because: You are the assignee for the bug.
[Bug 152336] Improve CSV field separator detection/behavior when several separators are selected on import
https://bugs.documentfoundation.org/show_bug.cgi?id=152336 --- Comment #13 from Mike Kaganski --- (In reply to m_a_riosv from comment #12) This is a very important issue. Please file it as a regression bug. Gabriel: please consider making this an opt-in (an option in filter option; a "detect me" kind of a button in the dialog...) What do you think? -- You are receiving this mail because: You are the assignee for the bug.
[Bug 152336] Improve CSV field separator detection/behavior when several separators are selected on import
https://bugs.documentfoundation.org/show_bug.cgi?id=152336 m_a_riosv changed: What|Removed |Added CC||miguelangelrv@libreoffice.o ||rg --- Comment #12 from m_a_riosv --- Created attachment 193559 --> https://bugs.documentfoundation.org/attachment.cgi?id=193559=edit Sample file csv With Version: 24.8.0.0.alpha0+ (X86_64) / LibreOffice Community Build ID: 5ebdcc60701ca5f2d267f92fa69ac06202309c52 CPU threads: 16; OS: Windows 10.0 Build 22631; UI render: Skia/Raster; VCL: win Locale: es-ES (es_ES); UI: en-US Calc: CL threaded The issue is that now, locale setting is not preserved in the import window. As was the case until 24.2. Seems it is reset to the default locale in my case es-ES. -- You are receiving this mail because: You are the assignee for the bug.
[Bug 152336] Improve CSV field separator detection/behavior when several separators are selected on import
https://bugs.documentfoundation.org/show_bug.cgi?id=152336 --- Comment #11 from Commit Notification --- Gabriel Masei committed a patch related to this issue. It has been pushed to "master": https://git.libreoffice.org/core/commit/2feda8ba21acdcf33a9b4ba94742f574c17839bd tdf#152336 Detect charset and separators for csv files It will be available in 24.8.0. The patch should be included in the daily builds available at https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More information about daily builds can be found at: https://wiki.documentfoundation.org/Testing_Daily_Builds Affected users are encouraged to test the fix and report feedback. -- You are receiving this mail because: You are the assignee for the bug.
[Bug 152336] Improve CSV field separator detection/behavior when several separators are selected on import
https://bugs.documentfoundation.org/show_bug.cgi?id=152336 Commit Notification changed: What|Removed |Added Whiteboard||target:24.8.0 -- You are receiving this mail because: You are the assignee for the bug.
[Bug 152336] Improve CSV field separator detection/behavior when several separators are selected on import
https://bugs.documentfoundation.org/show_bug.cgi?id=152336 --- Comment #10 from Mike Kaganski --- (In reply to Eyal Rozenberg from comment #9) > I would assume that if I save a file in a certain format, the specifics > would be independent of the input filtering, and only respect the format's > spec (and whatever defaults I have chosen the app to abide by if the spec is > flexible). Thus I would expect a TSV to be saved with tabs and a CSV with > commas even if, say, the input was parsed so as to separate fields by spaces. This may be a preference of some; but I claim, that at least for CSV, this would be a minority's preference; and the majority-to-minority ratio here will be HUGE. Most people who use Calc to edit CSVs will need Calc to retain the original file's structure; especially taking into account, that all "specs" in CSV are application-specific (and there is a *informational memo* RFC, reflecting some suggestions, but not binding). "Application-specific" here relates to the workflow (generation/consuming applications), not to the editing application in the middle. And a workflow of "normalization" of heterogenously-generated CSVs will be a minority of use cases. -- You are receiving this mail because: You are the assignee for the bug.
[Bug 152336] Improve CSV field separator detection/behavior when several separators are selected on import
https://bugs.documentfoundation.org/show_bug.cgi?id=152336 --- Comment #9 from Eyal Rozenberg --- I would assume that if I save a file in a certain format, the specifics would be independent of the input filtering, and only respect the format's spec (and whatever defaults I have chosen the app to abide by if the spec is flexible). Thus I would expect a TSV to be saved with tabs and a CSV with commas even if, say, the input was parsed so as to separate fields by spaces. Now, I do remember there's a dialog before saving happens; but at that dialog, I would still expect LO to default to how the format is defined, e.g. commas for CSV, and ignore what the input had. Also, I'm not sure I see how "Detect separator for CSV files" is a dupe of this bug. -- You are receiving this mail because: You are the assignee for the bug.
[Bug 152336] Improve CSV field separator detection/behavior when several separators are selected on import
https://bugs.documentfoundation.org/show_bug.cgi?id=152336 --- Comment #8 from V Stuart Foote --- (In reply to Gabriel Masei from comment #6) OP from bug 160355 dupe'd here: A CSV (comma-separated values) file is (in theory) a file that stores tabular data in plain text using commas to separate values and newlines to separate records. There is a standard (RFC 4180) for CSV files format. However, there is a lack of adherence to this standard with multiple formats used instead. The most common part of the format that differs is the separator. Instead of comma, multiple other separators are used: semicolon, pipe, tab, space, ... Not knowing the structure of a CSV file makes difficult importing/converting it in LibreOffice. 1. In case of importing a CSV file an Import dialog is shown to the user where it can provide the right filter options for the format of the data. A default set of values for these options is provided when the dialog loads. This is a reasonable way of handling the issue. 2. Also, in case of conversions (performed without UI) Libreoffice provides the "infilter" parameter which is equivalent to the Import dialog from the above case. If the parameter is missing then some default values are used. 3. Although the above cases are handled reasonably, there is a third case which needs a better handling: automatic conversions where the format of the input file is not fixed, it can change from one file to another. In this case either a provided set of options through the "infilter" parameter is used or the default one. However, this will generate wrong conversions if the format differs from one file to another. A better approach is needed. Taking into account the above considerations I consider that some kind of "detection/guess" mechanism can be implement so that it will cover automatically a greater number of formats. And I'm talking especially about the separator. I already provided a patch for this here: https://gerrit.libreoffice.org/c/core/+/164936 . It first detects the character set and then the separator based on the detected character set. Also, it allows a small room for not well-formatted files. This detection applies to conversions as well as to the Import dialog as an initial suggestion. Hi Gabriel, thanks for the commit. You should probably tag your commit with the BZ issue, tdf#152336 -- You are receiving this mail because: You are the assignee for the bug.
[Bug 152336] Improve CSV field separator detection/behavior when several separators are selected on import
https://bugs.documentfoundation.org/show_bug.cgi?id=152336 --- Comment #7 from V Stuart Foote --- *** Bug 160355 has been marked as a duplicate of this bug. *** -- You are receiving this mail because: You are the assignee for the bug.
[Bug 152336] Improve CSV field separator detection/behavior when several separators are selected on import
https://bugs.documentfoundation.org/show_bug.cgi?id=152336 --- Comment #6 from Gabriel Masei --- Please take a look here: https://bugs.documentfoundation.org/show_bug.cgi?id=160355 . It implements some heuristics for separator suggestion. -- You are receiving this mail because: You are the assignee for the bug.
[Libreoffice-bugs] [Bug 152336] Improve CSV field separator detection/behavior when several separators are selected on import
https://bugs.documentfoundation.org/show_bug.cgi?id=152336 Mike Kaganski changed: What|Removed |Added See Also||https://bugs.documentfounda ||tion.org/show_bug.cgi?id=15 ||5946 -- You are receiving this mail because: You are the assignee for the bug.
[Libreoffice-bugs] [Bug 152336] Improve CSV field separator detection/behavior when several separators are selected on import
https://bugs.documentfoundation.org/show_bug.cgi?id=152336 BogdanB changed: What|Removed |Added Status|UNCONFIRMED |NEW Ever confirmed|0 |1 CC||buzea.bog...@libreoffice.or ||g --- Comment #5 from BogdanB --- Changed to New based on +1 from Stuart, and duplicates. -- You are receiving this mail because: You are the assignee for the bug.
[Libreoffice-bugs] [Bug 152336] Improve CSV field separator detection/behavior when several separators are selected on import
https://bugs.documentfoundation.org/show_bug.cgi?id=152336 Mike Kaganski changed: What|Removed |Added See Also||https://bugs.documentfounda ||tion.org/show_bug.cgi?id=53 ||449 -- You are receiving this mail because: You are the assignee for the bug.
[Libreoffice-bugs] [Bug 152336] Improve CSV field separator detection/behavior when several separators are selected on import
https://bugs.documentfoundation.org/show_bug.cgi?id=152336 Mike Kaganski changed: What|Removed |Added CC||arshamaton...@gmail.com --- Comment #4 from Mike Kaganski --- *** Bug 127938 has been marked as a duplicate of this bug. *** -- You are receiving this mail because: You are the assignee for the bug.
[Libreoffice-bugs] [Bug 152336] Improve CSV field separator detection/behavior when several separators are selected on import
https://bugs.documentfoundation.org/show_bug.cgi?id=152336 Mike Kaganski changed: What|Removed |Added CC||oli...@westbook.eu --- Comment #3 from Mike Kaganski --- *** Bug 70111 has been marked as a duplicate of this bug. *** -- You are receiving this mail because: You are the assignee for the bug.
[Libreoffice-bugs] [Bug 152336] Improve CSV field separator detection/behavior when several separators are selected on import
https://bugs.documentfoundation.org/show_bug.cgi?id=152336 Mike Kaganski changed: What|Removed |Added See Also||https://bugs.documentfounda ||tion.org/show_bug.cgi?id=15 ||2337 -- You are receiving this mail because: You are the assignee for the bug.
[Libreoffice-bugs] [Bug 152336] Improve CSV field separator detection/behavior when several separators are selected on import
https://bugs.documentfoundation.org/show_bug.cgi?id=152336 V Stuart Foote changed: What|Removed |Added CC||vsfo...@libreoffice.org --- Comment #2 from V Stuart Foote --- +1 -- You are receiving this mail because: You are the assignee for the bug.
[Libreoffice-bugs] [Bug 152336] Improve CSV field separator detection/behavior when several separators are selected on import
https://bugs.documentfoundation.org/show_bug.cgi?id=152336 Mike Kaganski changed: What|Removed |Added CC||james@gmail.com --- Comment #1 from Mike Kaganski --- *** Bug 152332 has been marked as a duplicate of this bug. *** -- You are receiving this mail because: You are the assignee for the bug.
