another quesiton.... if you are using a database that only insert and
select (no delete/update) , you can use a more optimized database engine,
for example, you don't need to
begin
insert
commit
you can only insert, each insert is a 'automatic commit', this reduce qps,
other thing, instead of a jornaled engine with 1gb of jornal, you can use
10mb of jornal, just another idea, another idea is, you don't need a
transaction engine to do this, just a crash safe, in other words, you just
need to insert with double commit (crash safe jornal + data), you don't
need a full read-repetable or serialized or read-commit database, just a
single read-uncommited is ok, it's very high frequency qps, just one idea,
maybe you need a full transation database, but i don't see why you should
one big gun to kill this problem
what database are you using, and how you are reading and writing?
2014-11-23 20:44 GMT-02:00 Roberto Spadim <robe...@spadim.com.br>:
> well... if you are using a time based table, something like
> table(
> datetime,
> channel,
> value,
> primary (datetime,channel)
> )
>
> you can use some datetime rounding, for example
> datetime = floor( microtime() / 0.5) *0.5
>
> each 0,5 seconds you will get a new row, just an idea, there's more than
> this you can use to save space and writes
>
>
>
> 2014-11-23 20:28 GMT-02:00 Jerry Scharf <sch...@lagunawayconsulting.com>:
>
> Roberto,
>>
>> The problem comes from wanting to use a classic MVC web system. In these
>> systems, the assumption is that the database is the true state and that the
>> web draws whatever is in the database. Since I am scanning each of the
>> power supplies several times a second, this means that the database
>> journal(log) file is constantly getting update transactions (different than
>> the filesystem journal file.)
>>
>> After a night of thinking about it, I think I can cut the writes down by
>> a couple orders of magnitude. With that I think I can make it by.
>>
>> jerry
>>
>>
>> On 11/23/2014 03:53 AM, Roberto Spadim wrote:
>>
>> I dom't undestood the problem
>> You will write how many times per second? What's the minimal period that
>> you need? Maybe a compact file should solve this problem
>>
>> Em domingo, 23 de novembro de 2014, Jerry Scharf <
>> sch...@lagunawayconsulting.com> escreveu:
>>
>>> Hi,
>>>
>>> I am using a flash only system to be the master of my system for testing
>>> 300 power supplies. I want to use a MVC database/web design. In a normal
>>> disk based system, I wouldn't give the fact that the state of the each
>>> power supplies and microcontroller card is being updated several times a
>>> second, a disk would easily keep up and the wear problem is measured in
>>> years of continuous read/write.
>>>
>>> I am worried about wearing out the flash with writing entire blocks when
>>> the transaction log files are only incrementing by hundreds of bytes.
>>> 100TB of writes sounds like a lot, but doing a 128kB write hundreds of
>>> times a second could wipe that out in less than a month by my
>>> calculations.
>>>
>>> Is this something to worry about? I have a work around, but it's a bit
>>> tricky and less than perfect.
>>>
>>> thanks,
>>> jerry
>>>
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
>>> from Actuate! Instantly Supercharge Your Business Reports and Dashboards
>>> with Interactivity, Sharing, Native Excel Exports, App Integration & more
>>> Get technology previously reserved for billion-dollar corporations, FREE
>>>
>>> http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
>>> _______________________________________________
>>> Owfs-developers mailing list
>>> Owfs-developers@lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/owfs-developers
>>>
>>
>>
>> --
>> Roberto Spadim
>> SPAEmpresarial
>> Eng. Automação e Controle
>>
>>
>>
>> ------------------------------------------------------------------------------
>> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
>> from Actuate! Instantly Supercharge Your Business Reports and Dashboards
>> with Interactivity, Sharing, Native Excel Exports, App Integration & more
>> Get technology previously reserved for billion-dollar corporations,
>> FREEhttp://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
>>
>>
>>
>> _______________________________________________
>> Owfs-developers mailing
>> listOwfs-developers@lists.sourceforge.nethttps://lists.sourceforge.net/lists/listinfo/owfs-developers
>>
>>
>>
>>
>> ------------------------------------------------------------------------------
>> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
>> from Actuate! Instantly Supercharge Your Business Reports and Dashboards
>> with Interactivity, Sharing, Native Excel Exports, App Integration & more
>> Get technology previously reserved for billion-dollar corporations, FREE
>>
>> http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
>> _______________________________________________
>> Owfs-developers mailing list
>> Owfs-developers@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/owfs-developers
>>
>>
>
>
> --
> Roberto Spadim
> SPAEmpresarial
> Eng. Automação e Controle
>
--
Roberto Spadim
SPAEmpresarial
Eng. Automação e Controle
------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
_______________________________________________
Owfs-developers mailing list
Owfs-developers@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/owfs-developers
