[Rails] Re: Upload UTF-8 encoded textfile

2014-07-18 Thread Ronald Fischer
I think, the idea of first reading the lines, and then use force_encoding on the strings, would not work for two reasons: 1. As I have experienced, I already get the exception on the first byte which has the high-bit set (i.e. is not 7-bit ASCII) 2. If a UTF8 encoded character contained 0x0d

[Rails] Re: Upload UTF-8 encoded textfile

2014-07-17 Thread Ronald Fischer
As far I understand this article, this related to Rails 3 and MySQL, and how to use UTF8 encoded data everywhere. I don't know about MySQL, but Rails 4 and Ruby 2 with SQLite don't suffer this problem: I didn't have any trouble, processing all kinds of Unicode characters with my application,

Re: [Rails] Re: Upload UTF-8 encoded textfile

2014-07-17 Thread Colin Law
On 17 July 2014 10:22, Ronald Fischer li...@ruby-forum.com wrote: As far I understand this article, this related to Rails 3 and MySQL, and how to use UTF8 encoded data everywhere. I don't know about MySQL, but Rails 4 and Ruby 2 with SQLite don't suffer this problem: I didn't have any trouble,

Re: [Rails] Re: Upload UTF-8 encoded textfile

2014-07-17 Thread Eric Saupe
Colin, That shows how to create a Tempfile with a given encoding but the question is when a user uploads a file through a form and Rails creates a Tempfile is there a way to indicate that it should always create those Tempfiles with a default encoding such as UTF-8? On Thursday, July 17, 2014

Re: [Rails] Re: Upload UTF-8 encoded textfile

2014-07-17 Thread Colin Law
On 17 July 2014 15:42, Eric Saupe ericsa...@gmail.com wrote: Colin, That shows how to create a Tempfile with a given encoding but the question is when a user uploads a file through a form and Rails creates a Tempfile is there a way to indicate that it should always create those Tempfiles with

Re: [Rails] Re: Upload UTF-8 encoded textfile

2014-07-17 Thread Eric Saupe
In that case is there something like a simple config change to be made in say config/application.rb that tells Rails how to encode created Tempfiles? If not would that be something that could/should be added to the Rails project itself? On Thursday, July 17, 2014 8:51:30 AM UTC-6, Colin Law

[Rails] Re: Upload UTF-8 encoded textfile

2014-07-16 Thread Ronald Fischer
Yes, it is, as I found by trial-and-error. Note that the object is not just a File, it is of class Tempfile. I think this is quite common when working with a Tempfile object. To make a Tempfile threadsafe, you have to combine the creation of the filename and the creation of the file into one

[Rails] Re: Upload UTF-8 encoded textfile

2014-07-16 Thread Eric Saupe
Does setting config.encoding = utf-8 in your config/application.rb help? You'd also need to add # encoding: UTF-8 to the top of your file. I was reading this http://craiccomputing.blogspot.com/2011/02/rails-utf-8-and-heroku.html which seems to discuss this problem. On Wednesday, July 16,

[Rails] Re: Upload UTF-8 encoded textfile

2014-07-15 Thread Ronald Fischer
In this case, it is pretty certain that ever file will contain UTF-8 characters, and in general, I think the cases are few where we can assume input to be represented by 7-bit-ASCII. What I do not know for sure is whether or not the file will have a BOM, but I think Ruby can figure this out

[Rails] Re: Upload UTF-8 encoded textfile

2014-07-15 Thread Eric Saupe
Gotcha. Is the file actually opened when the controller is entered? (That's an honest question I'm interested in how that works coming as an upload from a form) The way you've described, that I failed to understand the first time, to me seems like the best way but I'd be interested to see what

[Rails] Re: Upload UTF-8 encoded textfile

2014-07-14 Thread Eric Saupe
Maybe try this? http://stackoverflow.com/questions/5163339/write-and-read-a-file-with-utf-8-encoding Does it matter if every file is considered UTF-8 even if it never contains a UTF-8 character? On Monday, July 14, 2014 5:01:11 AM UTC-6, Ruby-Forum.com User wrote: My Rails application (Rails