Re: Range and poolTask

[moechofe via Digitalmars-d-learn]

On Monday, 6 June 2016 at 11:25:00 UTC, Rene Zwanenburg wrote:

Could you elaborate a bit?


Yes.

I have an InputRange and need to pass it throughout a couple of 
iteration and manipulation functions such as filter, map and 
finishing by grouping with fold. Like:


myrange
.filter!xxx
.map!yyy
.tee!zzz
.fold!www


This process take a lot of time, so I decide to put it in 
parallel. I did something like this:


Appender!www output;
foreach(iii; parallel(myrange))
{
only(iii)
.filter!xxx
.map!yyy
.tee!zzz
.copy(output)
}
output.fold!www;

only() will create a Range with one item in it. Also, I can't 
group directly. I

 need to use a second Range.

It works but it's not elegant as the non-working example above is.

Re: Range and poolTask

[moechofe via Digitalmars-d-learn]

On Monday, 6 June 2016 at 09:38:32 UTC, Rene Zwanenburg wrote:

http://dlang.org/phobos/std_parallelism.html#.TaskPool

Or, more specifically,

http://dlang.org/phobos/std_parallelism.html#.TaskPool.amap
http://dlang.org/phobos/std_parallelism.html#.TaskPool.map


The functions passed to map or amap take the type of the element 
of the range as argument, but not a range itself.

Range and poolTask

[moechofe via Digitalmars-d-learn]

I wonder if it is possible to write something like this:
---
// taskPool.distribute -- take a range and distribute entries to 
different threads.

dirEntries().distribute(function(R1,R2)(R1 from,R2 to){
from
.filter!xxx
.map!yyy
.tee!zzz(to);
})
.each!www;
---
This would be great.

Re: missing data with parallel and stdin

[moechofe via Digitalmars-d-learn]

On Monday, 23 May 2016 at 14:16:13 UTC, Jack Stouffer wrote:
Sounds like a data race problem. Use a lock on the file write 
operation and see if that helps.


Like this?:

synchronized(mutex) copy(source,dest);

That didn't solve anything.
What I observe is: when the process is slower, more files are 
copied.

missing data with parallel and stdin

[moechofe via Digitalmars-d-learn]

Hi, I write a script that take a list of files from STDIN, 
compute some stuff, and copy files with a new names.


I got 33k lines at input but got only 3k-5k in the destination 
folder.

This is not append if I remove the .parallel() function.

What did I do wrong?

void delegate(string source,string dest) handler;

if(use_symlink) handler = delegate(string s,string d){
symlink(s,d);
}; else handler = delegate(string s,string d){
copy(s,d);
};

foreach(entry; parallel(stdin.byLineCopy)) try
{
auto source = buildPath(static_path,entry);
auto md5 = digest!MD5(File(source).byChunk(64*1024));
auto hash = toHexString!(LetterCase.lower)(md5);
auto file = text(hash,'_',baseName(entry));
auto dest = buildPath(hashed_path,file);
handler(source,dest);
writeln(entry,' ',file);
}
catch(Exception e)
{
error("Couldn't read, hash or copy %s",entry);
}

Re: Call a function passed as template parameter.

[moechofe via Digitalmars-d-learn]

On Thursday, 3 September 2015 at 13:16:41 UTC, Adam D. Ruppe 
wrote:

http://arsdnet.net/dcode/book/chapter_08/11/caller.d
https://github.com/adamdruppe/arsd/blob/master/jsvar.d#L608


Thank you for these examples.

Call a function passed as template parameter.

[moechofe via Digitalmars-d-learn]

Hi,

I would like to create a template that take a function as 
template parameter, create an arguments list for it, fill it with 
some data and call the function.


void foo(uint a, string b)
{
  // ...
}

void bar(long a, long b, string c)
{
  // ...
}

call(alias F)(JSONValue j)
{
  // create arguments list
  // assign arguments with the value from j
  // call the function
}

void main()
{
  call!foo(parseJSON("[123, \"nice\"]"));
  call!bar(parseJson("[1,2,3]"));
}

I found interesting stuff like ParameterTypeTuple! and Tuple! but 
I'm not be able to make it work together.


How can I do that?

n00b problem with deimos.openssl AES_cbc_encrypt

[moechofe via Digitalmars-d-learn]

Hi,

I'm try to encrypt a string with AES_cbc_encrypt using the 
openssl bindings from 
https://github.com/D-Programming-Deimos/openssl


And the result I got is wrong.

---
ubyte[32] key
// ...fill key

ubyte[16] iv;
// ...fill iv;

AES_KEY aes_key;
AES_set_encrypt_key(cast(const(ubyte)*)key.ptr,256,aes_key);

ubyte[] test_in = cast(ubyte[])foo;

ubyte[] test_out = new ubyte[](test_in.length + AES_BLOCK_SIZE - 
(test_in.length % AES_BLOCK_SIZE));


AES_cbc_encrypt(
cast(const(ubyte)*) test_in.ptr,
test_out.ptr,
test_in.length,
aes_key,
iv.ptr,
AES_ENCRYPT);
---

What I'm doing wrong?