date:20150824

Re: [Rails-core] String Allocation and Memory Bloat in ActiveRecord Objects

2015-08-24 Thread alexander . maznev

Hello Richard,
Thanks for the response. Unfortunately I'm not sure how it's possible to do 
this without the Hash look-up. For example one might imagine adding
something like this to ActiveSupport::Callbacks (which would prevent more 
than one string from being permanently allocated):

@@_current_kind = nil

def get_frozen_callback(kind)
  if @@_current_kind == kind
@@_current_callback
  else
@@_current_kind = kind
@@_current_callback = "_#{kind}_callbacks".freeze
  end
end

The problem is that this is what a typical run actually looks like:

[1] pry(main)> Symbolie.all.to_a
in #run_callbacks -- self: 
#, 
kind: checkout
  Symbolie Load (2.6ms)  SELECT "symbolies".* FROM "symbolies"
in #run_callbacks -- self: #, kind: find
in #run_callbacks -- self: #, kind: initialize
in #run_callbacks -- self: #, kind: find
in #run_callbacks -- self: #, kind: initialize
in #run_callbacks -- self: #, kind: find
in #run_callbacks -- self: #, kind: initialize
in #run_callbacks -- self: #, kind: find
in #run_callbacks -- self: #, kind: initialize
in #run_callbacks -- self: #, kind: find
in #run_callbacks -- self: #, kind: initialize
in #run_callbacks -- self: #, kind: find
in #run_callbacks -- self: #, kind: initialize
in #run_callbacks -- self: #, kind: find
# etc.

The objects are initialized sequentially so without a hash the strings will 
still have to be allocated and deallocated on a per-object basis - since as 
far as I understand (I may be mistaken here?) it is not useful to freeze a 
dynamically-generated string in Ruby without storing the reference e.g.

[2] pry(main)> "hello world".freeze.object_id
=> 70185328616580
[3] pry(main)> "hello world".freeze.object_id
=> 70185328616580
[4] pry(main)> var = "world"
=> "world"
[5] pry(main)> "hello #{var}".freeze.object_id
=> 70185328477560
[6] pry(main)> "hello #{var}".freeze.object_id
=> 70185328139100

I think for all normal applications the memory use from the hash is 
entirely marginal (about the same as a previous_changes on a single AR 
object) and since all objects will re-use these calls it doesn't make a lot 
of sense to clear.Actually I think even if the plan was to clear, it may be 
quite difficult to do so - since callbacks are an ActiveSupport concern 
they should be generally usable without AR (in theory?) -- meaning that any 
class which extends the concern and calls the define_callbacks and 
run_callbacks methods should be capable of using them correctly. Perhaps an 
alternative would be to make the string freezing behaviour specific to the 
ActiveRecord extension of ActiveSupport callbacks, and guaranteeing that 
the callbacks hash is cleared after all callbacks are run (it would need to 
have transactional not object based context) and may be little messy and 
I'm really not sure clearing 10-30 strings from memory (potentially only 
until the next callback cycle re-allocates them) would justify that. 

I will make a pull request on github as per your suggestion. 

- Alex

On Sunday, August 23, 2015 at 11:47:35 PM UTC+8, richard schneeman wrote:
>
> I appreciate people pointing out slow spots. I'm not the most quallified 
> to speak about AR but I'll try to help with the proposed patch. You can get 
> a bit more speed by taking out the hash lookup:
>
> ```ruby
> require 'benchmark/ips'
>
> var = "world".freeze
> hash = { var => "hello #{var}".freeze}
>
> Benchmark.ips do |x|
>   x.report("hash") { hash[var] }
>   x.report("compare ") { var == "world".freeze ? "hello world".freeze : 
> "hello #{var}" }
>   x.report("static  ") { "hello world".freeze }
>   x.report("none") { "hello #{var}" }
> end
> ```
>
> Gives you:
>
>
> ```
> Calculating -
> hash90.986k i/100ms
> compare 98.282k i/100ms
> static 103.704k i/100ms
> none90.593k i/100ms
> -
> hash  4.567M (±19.0%) i/s - 22.201M
> compare   5.441M (±13.7%) i/s - 26.831M
> static7.160M (±22.9%) i/s - 34.222M
> none  3.215M (±14.8%) i/s - 15.763M
> ```
>
> The hash also introduces the possibility of a memory leak, if there's only 
> two values that ever hit that method, we're fine, but at that rate we could 
>  optimize those two string paths since it would be faster. If somehow 
> arbitrary data from a user supplied `params` gets in there, then the hash 
> could quickly take up memory until your app crashes. It might not happen 
> today, but maybe in the future new functionality gets added that exposes 
> this method to a public API and then we're toast. 
>
> Thanks for the benchmarking and for the patch. If you make a PR to 
> github.com/rails/rails we can talk about it more, ping me @schneems. We 
> can also have someone with more AR experience take a look. 
>
>
>
> ---
> Richard Schneeman 
>

Re: [Rails-core] Re: Always set id from database when creating new record

2015-08-24 Thread Matt Jones

On Fri, Aug 21, 2015 at 2:46 PM, masa331  wrote:

> I don't think it would. You set the id in model, it's passed to database,
> and then database returns it back again
>
>
Some of the database adapters require an additional round-trip to retrieve
the newly-inserted ID; this doesn't happen if an id was supplied at record
creation. Others (like MySQL) will fall back on the DB's built in "last
inserted ID" mechanism, which doesn't work if you are generating IDs
manually (`mysql_insert_id`, for instance, only gets a value if the id
column was AUTO_INCREMENT).

For more detail, keep tracing from where you quoted. `_create_record` calls
`ActiveRecord::Relation#insert`, here:

https://github.com/rails/rails/blob/337684fa28a3e8d55874d5740585710d0fa99ead/activerecord/lib/active_record/relation.rb#L41

which does two things:

* if the primary key is empty and the adapter wants to use prefetch, it
generates an ID

* explicitly selects the supplied (or generated) primary key value and
passes it as a separate argument to the `insert` method in the connection
adapter.

The default implementation of the connection adapter's `insert` method:

https://github.com/rails/rails/blob/337684fa28a3e8d55874d5740585710d0fa99ead/activerecord/lib/active_record/connection_adapters/abstract/database_statements.rb#L107

will automatically return the value passed in as `id_value`, if present.
The MySQL and Postgres versions appear to do this as well.

A workaround for your case would be database-specific; for instance, on
MySQL I'd recommend that you ensure the trigger sets `LAST_INSERT_ID` and
then extract the value from the DB directly (via `SELECT LAST_INSERT_ID()`)
in an `after_create`. Unfortunately, the ID is the one field that the
standard, DB-independent approach to dealing with trigger-supplied data
(calling `reload` after saving) won't work for. :)

--Matt Jones

Dne pátek 21. srpna 2015 17:26:49 UTC+2 masa331 napsal(a):
>
>> Hi guys,
>>
>> this is code from ActiveRecord::Persistance:
>>
>> def _create_record(attribute_names = self.attribute_names)
>> attributes_values = arel_attributes_with_values_for_create(
>> attribute_names)
>>   new_id = self.class.unscoped.insert attributes_values   self.id ||=
>> new_id if self.class.primary_key
>>   @new_record = false   id end
>>
>> I'm curious about the rational behind line no. 5. Why we set `self.id`
>> to value returned from db only if it was previously empty? I couldn't find
>> any test nor some discussion. Also the code dates back to 2011.
>>
>> I work with some legacy db where primary key is string which is modified
>> in some database trigger. Because the `self.id` is set before save it
>> isn't assigned to proper value returned from db after trigger modification.
>>
>> I can't think of any problem if we always assign id to value returned
>> from database.
>>
>> I would love to create PR if it's ok
>>
>> thanks!
>>
>> Sledujte náš blog.uol.cz, facebook.com/uol.cz, gplus.to/uolcz
>> 
>
>
> Sledujte náš blog.uol.cz, facebook.com/uol.cz, gplus.to/uolcz
> 
>
> --
> You received this message because you are subscribed to the Google Groups
> "Ruby on Rails: Core" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to rubyonrails-core+unsubscr...@googlegroups.com.
> To post to this group, send email to rubyonrails-core@googlegroups.com.
> Visit this group at http://groups.google.com/group/rubyonrails-core.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups "Ruby 
on Rails: Core" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to rubyonrails-core+unsubscr...@googlegroups.com.
To post to this group, send email to rubyonrails-core@googlegroups.com.
Visit this group at http://groups.google.com/group/rubyonrails-core.
For more options, visit https://groups.google.com/d/optout.

[Rails-core] [ANN] Rails 4.2.4 and 4.1.13 have been released!

2015-08-24 Thread Rafael Mendonça França

Hi everyone,

I am happy to announce that Rails 4.2.4 and 4.1.13 have been released.

No regressions were found so these releases include the same changes that
the release candidates.

## CHANGES since 4.1.12

To view the changes for each gem, please read the changelogs on GitHub:

* [Action Mailer CHANGELOG](
https://github.com/rails/rails/blob/v4.1.13/actionmailer/CHANGELOG.md)
* [Action Pack CHANGELOG](
https://github.com/rails/rails/blob/v4.1.13/actionpack/CHANGELOG.md)
* [Action View CHANGELOG](
https://github.com/rails/rails/blob/v4.1.13/actionview/CHANGELOG.md)
* [Active Model CHANGELOG](
https://github.com/rails/rails/blob/v4.1.13/activemodel/CHANGELOG.md)
* [Active Record CHANGELOG](
https://github.com/rails/rails/blob/v4.1.13/activerecord/CHANGELOG.md)
* [Active Support CHANGELOG](
https://github.com/rails/rails/blob/v4.1.13/activesupport/CHANGELOG.md)
* [Railties CHANGELOG](
https://github.com/rails/rails/blob/v4.1.13/railties/CHANGELOG.md)

*Full listing*

To see the full list of changes, [check out all the commits on
GitHub](https://github.com/rails/rails/compare/v4.1.12...v4.1.13).

## CHANGES since 4.2.3

To view the changes for each gem, please read the changelogs on GitHub:

* [Action Mailer CHANGELOG](
https://github.com/rails/rails/blob/v4.2.4/actionmailer/CHANGELOG.md)
* [Action Pack CHANGELOG](
https://github.com/rails/rails/blob/v4.2.4/actionpack/CHANGELOG.md)
* [Action View CHANGELOG](
https://github.com/rails/rails/blob/v4.2.4/actionview/CHANGELOG.md)
* [Active Model CHANGELOG](
https://github.com/rails/rails/blob/v4.2.4/activemodel/CHANGELOG.md)
* [Active Record CHANGELOG](
https://github.com/rails/rails/blob/v4.2.4/activerecord/CHANGELOG.md)
* [Active Support CHANGELOG](
https://github.com/rails/rails/blob/v4.2.4/activesupport/CHANGELOG.md)
* [Active Job CHANGELOG](
https://github.com/rails/rails/blob/v4.2.4/activejob/CHANGELOG.md)
* [Railties CHANGELOG](
https://github.com/rails/rails/blob/v4.2.4/railties/CHANGELOG.md)

*Full listing*

To see the full list of changes, [check out all the commits on
GitHub](https://github.com/rails/rails/compare/v4.2.3...v4.2.4).

## SHA-1

If you'd like to verify that your gem is the same as the one I've uploaded,
please use these SHA-1 hashes.

Here are the checksums for 4.1.13:

```
$ shasum *4.1.13*
7c637fed154a152451adb8ef1ebae136e0502ebd  actionmailer-4.1.13.gem
61303b7ee877aa06f16ce0ba6a7cce21ca031f42  actionpack-4.1.13.gem
39c5a4a290a5969402bf5c4ac61b99e5dcb40c3c  actionview-4.1.13.gem
6d88159aa15ea48f9118ece97ec549807a504f55  activemodel-4.1.13.gem
1107470b19cc0d6608d431678c78d3871c8df229  activerecord-4.1.13.gem
f79bdfa0105c1a833069d989291ec9d23bc84d92  activesupport-4.1.13.gem
6391b4a323e0b30118ffdbad6bb5f68459920d7b  rails-4.1.13.gem
fc64a4f89a3fa8f9ab2a9edd37d7338a79cb9859  railties-4.1.13.gem
```

Here are the checksums for 4.2.4:

```
$ shasum *4.2.4*
a866394bb4682afaa6d54458b1f3be0dca5e862e  actionmailer-4.2.4.gem
b625e58fb8fa11185ff81630f3bef5c517e294e4  actionpack-4.2.4.gem
334ddc60724e4e1ff99e2170d2e435805d0ade0b  actionview-4.2.4.gem
7bd46253f17bcff5c8ae3ba0f0938c38b0ca1e2f  activejob-4.2.4.gem
d361d1e0eac7851103ac2de4c9359a556c724ab9  activemodel-4.2.4.gem
d9faecb759827fc7944558463388efbbd9f20005  activerecord-4.2.4.gem
ab4283126725e3aa9c248e2e2119548ad205732d  activesupport-4.2.4.gem
67d7a12105f5389dd7b511eff5f56c6c048175d8  rails-4.2.4.gem
8e9eab180dbd97eaf7626601c55e61ea1ae68284  railties-4.2.4.gem
```

I'd like to thank you all, every contributor who helped with this release.

-- 
You received this message because you are subscribed to the Google Groups "Ruby 
on Rails: Core" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to rubyonrails-core+unsubscr...@googlegroups.com.
To post to this group, send email to rubyonrails-core@googlegroups.com.
Visit this group at http://groups.google.com/group/rubyonrails-core.
For more options, visit https://groups.google.com/d/optout.

Re: [Rails-core] Feature suggestion: Use symbols internally with HashWithIndifferentAccess

2015-08-24 Thread richard schneeman

In 2.2 the speed of symbols and strings are much closer:




```


require 'benchmark/ips'




hash = {}




Benchmark.ips do |x|

  x.report("symbol ") { hash[:symbol] }

  x.report("string ") { hash["string"]}

end

```




Gives us




```


Calculating -

             symbol    130.247k i/100ms

             string    123.185k i/100ms

-

             symbol       7.795M (±14.2%) i/s -     38.162M

             string       7.532M (±16.7%) i/s -     36.216M



```







If you want your HWIA to be faster, use strings instead of symbols. Check out 
http://www.sitepoint.com/unraveling-string-key-performance-ruby-2-2/ and scroll 
down to the part about Ruby 2.2













---Richard Schneeman


http://www.schneems.com

On Sun, Aug 23, 2015 at 8:23 PM, Rafael Mendonça França
 wrote:

> Thank you for the suggestion? Could you explain what advantages this change
> will bring? I think we can change but it is a backward incompatible change
> and we really a good reason to do so.
> On Sun, Aug 23, 2015, 11:39 Leslie Hoare  wrote:
>> Currently HashWithIndifferentAccess stores using string keys, and any
>> symbols passed in get converted to strings. As I understand it, this was
>> due to the fact that in earlier versions of Ruby, symbols would not get
>> garbage collected and would thus stick around for the lifetime of the
>> process.
>>
>> With Rails 5, and its minimum version requirement of Ruby 2.2.1, I don't
>> believe this to be an issue anymore, and symbols perform better as Hash
>> keys.
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Ruby on Rails: Core" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to rubyonrails-core+unsubscr...@googlegroups.com.
>> To post to this group, send email to rubyonrails-core@googlegroups.com.
>> Visit this group at http://groups.google.com/group/rubyonrails-core.
>> For more options, visit https://groups.google.com/d/optout.
>>
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Ruby on Rails: Core" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to rubyonrails-core+unsubscr...@googlegroups.com.
> To post to this group, send email to rubyonrails-core@googlegroups.com.
> Visit this group at http://groups.google.com/group/rubyonrails-core.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups "Ruby 
on Rails: Core" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to rubyonrails-core+unsubscr...@googlegroups.com.
To post to this group, send email to rubyonrails-core@googlegroups.com.
Visit this group at http://groups.google.com/group/rubyonrails-core.
For more options, visit https://groups.google.com/d/optout.

Re: [Rails-core] Feature suggestion: Use symbols internally with HashWithIndifferentAccess

2015-08-24 Thread Xavier Noria

On Mon, Aug 24, 2015 at 3:22 AM, Rafael Mendonça França <
rafaelmfra...@gmail.com> wrote:

> I think we can change but it is a backward incompatible change
>
In particular, it belongs to the public interface that keys are strings,
#keys, #each, etc. are guaranteed to return strings.

-- 
You received this message because you are subscribed to the Google Groups "Ruby 
on Rails: Core" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to rubyonrails-core+unsubscr...@googlegroups.com.
To post to this group, send email to rubyonrails-core@googlegroups.com.
Visit this group at http://groups.google.com/group/rubyonrails-core.
For more options, visit https://groups.google.com/d/optout.

Re: [Rails-core] String Allocation and Memory Bloat in ActiveRecord Objects

Re: [Rails-core] Re: Always set id from database when creating new record

[Rails-core] [ANN] Rails 4.2.4 and 4.1.13 have been released!

Re: [Rails-core] Feature suggestion: Use symbols internally with HashWithIndifferentAccess

Re: [Rails-core] Feature suggestion: Use symbols internally with HashWithIndifferentAccess

5 matches

Site Navigation

Mail list logo

Footer information