Re: [Factor-talk] Confused about run-pipeline
Well, there's your problem: you don't want `run-pipeline` at all. :)
IN: scratchpad "echo \"Hi\"" run-detached my-process set-global
IN: scratchpad my-process get-global .
T{ process
{ command "echo \"Hi\"" }
{ environment H{ } }
{ environment-mode +append-environment+ }
{ group +same-group+ }
{ status 0 }
}
`run-pipeline` is like a series of Unix pipes. So,
{ "cat log.txt" "grep error" "sort" "uniq" } run-pipeline
is like typing
$ cat log.txt | grep error | sort | uniq
at a bash prompt. Except not every element of the pipeline has to be a
process descriptor---you can insert Factor code along the way.
Roughly, it works by sequentially calling `run-pipeline-element` on each
item in your sequence. `run-pipeline-element` is defined by:
IN: scratchpad \ run-pipeline-element see-methods
USING: combinators destructors io io.pipes.private kernel
quotations ;
M: callable run-pipeline-element
[
[ [ ?reader ] [ ?writer ] bi* ] dip
[ ( -- result ) call-effect ] curry with-streams*
] with-destructors ;
USING: accessors destructors io.launcher io.pipes.private kernel
;
M: object run-pipeline-element
[ >process swap >>stdout swap >>stdin run-detached ]
[ drop [ [ dispose ] when* ] bi@ ] 3bi wait-for-process ;
I.e., it's already calling `run-detached` on anything in the pipeline
that's just a process description. Otherwise, it expects a quotation that
takes no inputs, produces one output, and then calls that quotation with
the I/O streams rebound. E.g.,
IN: scratchpad { "cat /tmp/patch" [ readln >string ] } run-pipeline .
{
0
"diff --git a/basis/prettyprint/prettyprint-tests.factor
b/basis/prettyprint/prettyprint-tests.factor"
}
We see that the result of the first item in the pipeline is 0 (the exit
code of cat), and the result of the second item in the pipeline the output
from the Factor quotation, which it read from standard-input. Except that
standard-input was bound to the output of the prior pipeline element (the
output of cat).
Hope that helps,
--Alex Vondrak
--
Precog is a next-generation analytics platform capable of advanced
analytics on semi-structured data. The platform includes APIs for building
apps and a phenomenal toolset for data science. Developers can use
our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter___
Factor-talk mailing list
Factor-talk@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/factor-talk
Re: [Factor-talk] Confused about run-pipeline
Hi -
Yes, indeed, the run-pipeline example works fine, composed
of strings of unix commands. My components are Factor callable
quotations and a launch descriptor:
SYMBOL: my-process
{
! ( -- desc )
[ run-detached ]! ( -- process )
[ my-process set-global t ] ! ( -- t )
}
run-pipeline
Calling run-pipeline on the sequence fails with:
"Error in thread 341 (Future, [ ~slice~ ~quotation~ ~quotation~ dip
run-pipeline-element...):
Data stack underflow"
Note that the phrase " run-detached my-process set-global t"
completes successfully in the listener. Also, the code compiles without
error in its
compilation unit.
I can step down through the code as far as the word map! successfully.
However,
trying to set a breakpoint on map! hangs the system, so I can't follow into
that word.
I added the boolean at the end to satisfy the requirement that each
component
must output a single value. I can't see how that alone would produce the
stack
underflow.
Since the vocabulary description for run-pipeline reads:
"Pipeline components must be one of the following:
• A quotation. The quotation is called with both input-stream and
output-stream rebound, except for the first and last pipeline components,
and it must output a single value.
• A process launch descriptor. See Launch descriptors." ,
I suspect my problem is in the binding of the input/output streams. I'll
experiment
with that, but do let me know if I'm missing something simple. I just want
to find
out how to properly use run-pipeline with ordinary callables, especially
quotations
composed of words with stack effect ( -- ).
Thanks for the replies.
~cw
On Mon, Apr 22, 2013 at 6:20 AM, John Benediktsson wrote:
> The example is simple and works fine:
>
> { "cat log.txt" "grep error" "sort" "uniq" } run-pipeline
>
> Can you explain the more complicated thing you're trying to do?
>
>
> On Mon, Apr 22, 2013 at 4:30 AM, CW Alston wrote:
>
>> Factor folks -
>>
>>
>> I'm missing something trying to properly set up the components
>>
>> of a sequence to be passed to run-pipeline. I intend the components
>>
>> to be:
>>
>> (1) a process launch descriptor, with stack effect ( -- desc ) ;
>>
>> (2) a quotation composed only of run-detached, stack effect ( desc --
>> process ) ;
>>
>> (3) a quotation to store the proc in a variable, stack effect ( process
>> -- );
>>
>> -these followed by quotations of words with stack effect ( -- ),
>>
>> all duly wrapped in a sequence { … }.
>>
>>
>> The word description for run-pipeline says that run-pipeline:
>>
>> "Creates a pipe between each pipeline component, with the output of each
>> component becoming the input of the next.
>>
>>
>> The first component reads input from input-stream and the last component
>> writes output to output-stream.
>>
>>
>> Each component runs in its own thread, and the word returns when all
>> components finish executing. Each component outputs a result value."
>>
>>
>> My sundry attempts to meet these requirements typically result in a
>>
>> data stack underflow on execution, or an aberrant stack effect during
>> compile.
>>
>>
>> Can anyone sketch a proper set-up for generic requirements like mine,
>>
>> to successfully pass a component sequence to run-pipeline?
>>
>>
>> Thanks for any suggestions,
>>
>> ~cw
>>
>>
>> --
>> *~ Memento Amori*
>>
>>
>> --
>> Precog is a next-generation analytics platform capable of advanced
>> analytics on semi-structured data. The platform includes APIs for building
>> apps and a phenomenal toolset for data science. Developers can use
>> our toolset for easy data analysis & visualization. Get a free account!
>> http://www2.precog.com/precogplatform/slashdotnewsletter
>> ___
>> Factor-talk mailing list
>> Factor-talk@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/factor-talk
>>
>>
>
--
*~ Memento Amori*
--
Precog is a next-generation analytics platform capable of advanced
analytics on semi-structured data. The platform includes APIs for building
apps and a phenomenal toolset for data science. Developers can use
our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter___
Factor-talk mailing list
Factor-talk@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/factor-talk
Re: [Factor-talk] Idiomatic use of sequences for iterative functions?
Thank you, John. That's what I was looking for. I must say: currying an entire sequence is pretty wild! It's a non-obvious solution. >> "I'm not sure that is undesirable here...?" No, it's not undesirable, except for the fact that 30-40% of that code exists for the sake of the side-effect. Of course, computation by calculation is shorter again ... USING: arrays math math.ranges math.primes.factors sequences ; IN: lamps : ( -- seq ) 100 [1,b] >array ; : calc-switching ( -- lamps ) [ divisors length even? ] map ; (But then, that wasn't my purpose. I was trying to figure out how to get a nested iteration done with sequences instead of always falling back to recursion.) Thanks, again. Loryn Jenkins On Tue, Apr 23, 2013 at 12:44 AM, John Benediktsson wrote: > Something like this would be simpler: > > http://paste.factorcode.org/paste?id=2914#1329 > > You don't need a symbol to pass the variable around, and even though it is > "side-effecting", I'm not sure that is undesirable here...? > > > On Mon, Apr 22, 2013 at 7:17 AM, Loryn Jenkins wrote: > >> Hi >> >> In "Factor Philosophy", slava writes: "Imperative programming and indexed >> loops are almost always not the most idiomatic solution." >> >> I've implemented this puzzle ( >> http://mindyourdecisions.com/blog/2013/04/22/100-light-bulbs-puzzle/ ) >> using a side-effecting function: >> http://paste.factorcode.org/paste?id=2914 >> >> The words do-switching and toggle-switches implement the equivalent of a >> nested loop. >> >> Without using recursion, counted for loops or side-effecting functions, >> what is an idiomatic way of implementing this nested iteration? >> >> Loryn Jenkins >> >> >> -- >> Precog is a next-generation analytics platform capable of advanced >> analytics on semi-structured data. The platform includes APIs for building >> apps and a phenomenal toolset for data science. Developers can use >> our toolset for easy data analysis & visualization. Get a free account! >> http://www2.precog.com/precogplatform/slashdotnewsletter >> ___ >> Factor-talk mailing list >> Factor-talk@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/factor-talk >> >> > > > -- > Precog is a next-generation analytics platform capable of advanced > analytics on semi-structured data. The platform includes APIs for building > apps and a phenomenal toolset for data science. Developers can use > our toolset for easy data analysis & visualization. Get a free account! > http://www2.precog.com/precogplatform/slashdotnewsletter > ___ > Factor-talk mailing list > Factor-talk@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/factor-talk > > -- Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis & visualization. Get a free account! http://www2.precog.com/precogplatform/slashdotnewsletter___ Factor-talk mailing list Factor-talk@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/factor-talk
Re: [Factor-talk] Idiomatic use of sequences for iterative functions?
Something like this would be simpler: http://paste.factorcode.org/paste?id=2914#1329 You don't need a symbol to pass the variable around, and even though it is "side-effecting", I'm not sure that is undesirable here...? On Mon, Apr 22, 2013 at 7:17 AM, Loryn Jenkins wrote: > Hi > > In "Factor Philosophy", slava writes: "Imperative programming and indexed > loops are almost always not the most idiomatic solution." > > I've implemented this puzzle ( > http://mindyourdecisions.com/blog/2013/04/22/100-light-bulbs-puzzle/ ) > using a side-effecting function: http://paste.factorcode.org/paste?id=2914 > > The words do-switching and toggle-switches implement the equivalent of a > nested loop. > > Without using recursion, counted for loops or side-effecting functions, > what is an idiomatic way of implementing this nested iteration? > > Loryn Jenkins > > > -- > Precog is a next-generation analytics platform capable of advanced > analytics on semi-structured data. The platform includes APIs for building > apps and a phenomenal toolset for data science. Developers can use > our toolset for easy data analysis & visualization. Get a free account! > http://www2.precog.com/precogplatform/slashdotnewsletter > ___ > Factor-talk mailing list > Factor-talk@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/factor-talk > > -- Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis & visualization. Get a free account! http://www2.precog.com/precogplatform/slashdotnewsletter___ Factor-talk mailing list Factor-talk@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/factor-talk
[Factor-talk] Idiomatic use of sequences for iterative functions?
Hi In "Factor Philosophy", slava writes: "Imperative programming and indexed loops are almost always not the most idiomatic solution." I've implemented this puzzle ( http://mindyourdecisions.com/blog/2013/04/22/100-light-bulbs-puzzle/ ) using a side-effecting function: http://paste.factorcode.org/paste?id=2914 The words do-switching and toggle-switches implement the equivalent of a nested loop. Without using recursion, counted for loops or side-effecting functions, what is an idiomatic way of implementing this nested iteration? Loryn Jenkins -- Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis & visualization. Get a free account! http://www2.precog.com/precogplatform/slashdotnewsletter___ Factor-talk mailing list Factor-talk@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/factor-talk
Re: [Factor-talk] Confused about run-pipeline
The example is simple and works fine:
{ "cat log.txt" "grep error" "sort" "uniq" } run-pipeline
Can you explain the more complicated thing you're trying to do?
On Mon, Apr 22, 2013 at 4:30 AM, CW Alston wrote:
> Factor folks -
>
>
> I'm missing something trying to properly set up the components
>
> of a sequence to be passed to run-pipeline. I intend the components
>
> to be:
>
> (1) a process launch descriptor, with stack effect ( -- desc ) ;
>
> (2) a quotation composed only of run-detached, stack effect ( desc --
> process ) ;
>
> (3) a quotation to store the proc in a variable, stack effect ( process
> -- );
>
> -these followed by quotations of words with stack effect ( -- ),
>
> all duly wrapped in a sequence { … }.
>
>
> The word description for run-pipeline says that run-pipeline:
>
> "Creates a pipe between each pipeline component, with the output of each
> component becoming the input of the next.
>
>
> The first component reads input from input-stream and the last component
> writes output to output-stream.
>
>
> Each component runs in its own thread, and the word returns when all
> components finish executing. Each component outputs a result value."
>
>
> My sundry attempts to meet these requirements typically result in a
>
> data stack underflow on execution, or an aberrant stack effect during
> compile.
>
>
> Can anyone sketch a proper set-up for generic requirements like mine,
>
> to successfully pass a component sequence to run-pipeline?
>
>
> Thanks for any suggestions,
>
> ~cw
>
>
> --
> *~ Memento Amori*
>
>
> --
> Precog is a next-generation analytics platform capable of advanced
> analytics on semi-structured data. The platform includes APIs for building
> apps and a phenomenal toolset for data science. Developers can use
> our toolset for easy data analysis & visualization. Get a free account!
> http://www2.precog.com/precogplatform/slashdotnewsletter
> ___
> Factor-talk mailing list
> Factor-talk@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/factor-talk
>
>
--
Precog is a next-generation analytics platform capable of advanced
analytics on semi-structured data. The platform includes APIs for building
apps and a phenomenal toolset for data science. Developers can use
our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter___
Factor-talk mailing list
Factor-talk@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/factor-talk
Re: [Factor-talk] Confused about run-pipeline
i guess you wand to use run-pipeline on factor quotes instead of just unix
commands so you would dispatch to run-pipeline-element on a callable
instead of a "object" which i guess is a string (?)
i would just throw a quote instead of a string into the sequence and it
should just work as long as run-pipeline creates input/outputs for it, your
quotation would though need to read stdin and write stdout i guess so it
would "fit in"
On Mon, Apr 22, 2013 at 1:30 PM, CW Alston wrote:
> Factor folks -
>
>
> I'm missing something trying to properly set up the components
>
> of a sequence to be passed to run-pipeline. I intend the components
>
> to be:
>
> (1) a process launch descriptor, with stack effect ( -- desc ) ;
>
> (2) a quotation composed only of run-detached, stack effect ( desc --
> process ) ;
>
> (3) a quotation to store the proc in a variable, stack effect ( process
> -- );
>
> -these followed by quotations of words with stack effect ( -- ),
>
> all duly wrapped in a sequence { … }.
>
>
> The word description for run-pipeline says that run-pipeline:
>
> "Creates a pipe between each pipeline component, with the output of each
> component becoming the input of the next.
>
>
> The first component reads input from input-stream and the last component
> writes output to output-stream.
>
>
> Each component runs in its own thread, and the word returns when all
> components finish executing. Each component outputs a result value."
>
>
> My sundry attempts to meet these requirements typically result in a
>
> data stack underflow on execution, or an aberrant stack effect during
> compile.
>
>
> Can anyone sketch a proper set-up for generic requirements like mine,
>
> to successfully pass a component sequence to run-pipeline?
>
>
> Thanks for any suggestions,
>
> ~cw
>
>
> --
> *~ Memento Amori*
>
>
> --
> Precog is a next-generation analytics platform capable of advanced
> analytics on semi-structured data. The platform includes APIs for building
> apps and a phenomenal toolset for data science. Developers can use
> our toolset for easy data analysis & visualization. Get a free account!
> http://www2.precog.com/precogplatform/slashdotnewsletter
> ___
> Factor-talk mailing list
> Factor-talk@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/factor-talk
>
>
--
Precog is a next-generation analytics platform capable of advanced
analytics on semi-structured data. The platform includes APIs for building
apps and a phenomenal toolset for data science. Developers can use
our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter___
Factor-talk mailing list
Factor-talk@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/factor-talk
[Factor-talk] Confused about run-pipeline
Factor folks -
I'm missing something trying to properly set up the components
of a sequence to be passed to run-pipeline. I intend the components
to be:
(1) a process launch descriptor, with stack effect ( -- desc ) ;
(2) a quotation composed only of run-detached, stack effect ( desc --
process ) ;
(3) a quotation to store the proc in a variable, stack effect ( process --
);
-these followed by quotations of words with stack effect ( -- ),
all duly wrapped in a sequence { … }.
The word description for run-pipeline says that run-pipeline:
"Creates a pipe between each pipeline component, with the output of each
component becoming the input of the next.
The first component reads input from input-stream and the last component
writes output to output-stream.
Each component runs in its own thread, and the word returns when all
components finish executing. Each component outputs a result value."
My sundry attempts to meet these requirements typically result in a
data stack underflow on execution, or an aberrant stack effect during
compile.
Can anyone sketch a proper set-up for generic requirements like mine,
to successfully pass a component sequence to run-pipeline?
Thanks for any suggestions,
~cw
--
*~ Memento Amori*
--
Precog is a next-generation analytics platform capable of advanced
analytics on semi-structured data. The platform includes APIs for building
apps and a phenomenal toolset for data science. Developers can use
our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter___
Factor-talk mailing list
Factor-talk@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/factor-talk
