Hi Achim,
A Dimecres 01 Novembre 2006 22:52, Achim Gaedke escrigué:
> Hi Francesc!
>
> This big question is still open to me. We were playing around with
> compression methods, chunk sizes and so on. But basicaly this was a
> "production system". That's why I am working hard on that issue to avoid
> this failure in future. (This was with python2.4.3 and pytables-1.3.2)
>
> By the way, using python2.4.4 and pytables-1.3.3 I found a warning, that
> was new to me:
>
> /home/achim/local/lib/python2.4/site-packages/tables/Table.py:1971:
> PerformanceWarning: table ``/data_pool/dict_Echo_Height__pos_`` is being
> preempted from alive nodes without its buffers being flushed. This may
> lead to very ineficient use of resources and even to fatal errors in
> certain situations. Please, do a call to the .flush() method on this
> table before start using other nodes.
> Caller was write_hdf5:130 in file
> /home/achim/damaris/frontends/reddamaris/DataPool.py.
> PerformanceWarning)
>
> I do not understand this:
> What means "preempted from alive nodes"?
This means that the table has been removed from the set of alive nodes
(that is, the table gets unreferenced in your program) before being
flushed. The message is telling you that you should call to
Table.flush() before dereferencing it.
> Does "this table" refer to dict_Echo_Height__pos_ or to the owners of
> "alive nodes"
"this table" refers to ``/data_pool/dict_Echo_Height__pos_``.
>
> The node works with a table of 10 or 20 rows only.
>
> h5_table_format= {
> "x" : tables.Float64Col(),
> "y_mean" : tables.Float64Col(),
> "y_sigma" : tables.Float64Col(),
> "n" : tables.Int64Col()
> }
>
> mr_table=hdffile.createTable(where=where,name=name,
> description=h5_table_format,
> title=title,
> expectedrows=len(self))
> mr_table.attrs.damaris_type="MeasurementResult"
> self.lock.acquire()
> try:
> mr_table.attrs.quantity_name=self.quantity_name
>
> row=mr_table.row
> for x in self.get_xdata():
> y=UserDict.UserDict.__getitem__(self,x)
> row["x"]=x
> if type(y) is types.FloatType:
> row["y_mean"]=y
> row["y_sigma"]=0.0
> row["n"]=1
> else:
> row["y_mean"]=y.mean()
> row["y_sigma"]=y.mean_sigma()
> row["n"]=y.n
> row.append()
>
> finally:
> #mr_table.flush()
> self.lock.release()
>
> This is the code. Till now I thought, there is no necessity for a
> flush... ReReading the manual I fear, I did make a misstake. I added the
> comment line and the warning vanished (of course removing the #).
Yeah, I've added the PerformanceWarning in 1.3.3 to warn people that
doesn't make a call to Table.flush() prior to dereference it. Doing
such a .flush() call is very important because this operation flushes
all buffers in memory to disk, making your programs safer and far less
memory hungry.
>
> Is it possible that both errors were related?
The PerformanceWarning and creating a leaf without a "FLAVOR"
attribute? uh, no, I don't think so.
Cheers,
--
>0,0< Francesc Altet http://www.carabos.com/
V V Cárabos Coop. V. Enjoy Data
"-"
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
Pytables-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/pytables-users
