Re: [julia-users] Pipeline interoperability with IOStream and IOBuffer

[William Wong]

Hi Steven,

Thank you very much for taking the time to try to try to answer my 
question. 

Perhaps if it is not clear enough, yes I have a usage question that I am 
asking here.  I'll try to rephrase it. Could you kindly answer this?

I have data (could be a string) that is stored in IOBuffer. I want to send 
it into a process, i.e. via a pipeline.  Is there a way to turn a string or 
any byte array into a stream? An example of what I want is in the code 
snipped in OP.

For example, java has something like 
this 
http://stackoverflow.com/questions/247161/how-do-i-turn-a-string-into-a-stream-in-java

I understand (as stated in the github issue) IOBuffer is not directly 
supported in pipelines (although the docs clearly say otherwise) and if you 
feel like that should be a discussion on the issue, then I will continue 
that particular discussion on the github issue. Just wanted to note that 
the devs there already dismissed this feature request as not necessary and 
labelled it a "documentation issue".

Thank you,
Will


On Friday, July 22, 2016 at 6:37:06 PM UTC-4, Steven G. Johnson wrote:
>
>
>
> On Friday, July 22, 2016 at 8:55:44 AM UTC-4, Tom Breloff wrote:
>>
>> To be fair, he was told, in that issue, that julia-users was a more 
>> appropriate place to ask this type of question. 
>
>
> No, he was told to ask on julia-users about usage questions — in this 
> case, if he can't figure out how to write from an IOBuffer to an IOStream 
> or to a process.  (No changes to Base are required to do this.)
>
> In contrast, if you want to continue to press for changes to the Julia 
> standard library, the correct forum is the Github issue.
>
>
>

Re: [julia-users] Pipeline interoperability with IOStream and IOBuffer

[Steven G. Johnson]

On Friday, July 22, 2016 at 8:55:44 AM UTC-4, Tom Breloff wrote:
>
> To be fair, he was told, in that issue, that julia-users was a more 
> appropriate place to ask this type of question. 


No, he was told to ask on julia-users about usage questions — in this case, 
if he can't figure out how to write from an IOBuffer to an IOStream or to a 
process.  (No changes to Base are required to do this.)

In contrast, if you want to continue to press for changes to the Julia 
standard library, the correct forum is the Github issue.

Re: [julia-users] Pipeline interoperability with IOStream and IOBuffer

[William Wong]

To add to that, this discussion had actually initially started out on 
Julia-Users
https://groups.google.com/forum/#!searchin/julia-users/pipeline$20iostream/julia-users/-66VWszB-mM/b9wno137BgAJ

I was also hoping that someone could help me with this question here:

> In either case, it seems that I need an IOStream to pipe data into a 
> pipeline command. How can I turn data into a stream that is not a file?
> https://groups.google.com/forum/#!msg/julia-users/R-F3F97leh4/o4zKINZbbvUJ 
> asks 
> a similar question but readall produces a string, I still need to stream it 
> into a pipeline command.
>

On Friday, July 22, 2016 at 8:55:44 AM UTC-4, Tom Breloff wrote:
>
> To be fair, he was told, in that issue, that julia-users was a more 
> appropriate place to ask this type of question. 
>
> On Friday, July 22, 2016, Steven G. Johnson  > wrote:
>
>>
>>
>> On Thursday, July 21, 2016 at 6:09:12 PM UTC-4, William Wong wrote:
>>>
>>> I'm trying to continue the discussion of 
>>> https://github.com/JuliaLang/julia/issues/15479
>>>
>>
>> The github issue is the appropriate place for discussion; it's not 
>> helpful to split the discussion into two separate forums. 
>>
>> (If you want to see progress on an issue, the most effective way is to 
>> submit a pull request attempting to fix the issue.)
>>
>

Re: [julia-users] Pipeline interoperability with IOStream and IOBuffer

[Tom Breloff]

To be fair, he was told, in that issue, that julia-users was a more
appropriate place to ask this type of question.

On Friday, July 22, 2016, Steven G. Johnson  wrote:

>
>
> On Thursday, July 21, 2016 at 6:09:12 PM UTC-4, William Wong wrote:
>>
>> I'm trying to continue the discussion of
>> https://github.com/JuliaLang/julia/issues/15479
>>
>
> The github issue is the appropriate place for discussion; it's not helpful
> to split the discussion into two separate forums.
>
> (If you want to see progress on an issue, the most effective way is to
> submit a pull request attempting to fix the issue.)
>

[julia-users] Pipeline interoperability with IOStream and IOBuffer

[William Wong]

Hello, 

I'm trying to continue the discussion of 
https://github.com/JuliaLang/julia/issues/15479

julia> run(pipeline(IOBuffer("a xyz b"), `grep xyz`))
ERROR: MethodError: `uvtype` has no method matching 
uvtype(::Base.AbstractIOBuffer{Array{UInt8,1}})
 in _jl_spawn at process.jl:253
 in anonymous at process.jl:415
 in setup_stdio at process.jl:403
 in spawn at process.jl:414
 in spawn at process.jl:293
 in run at process.jl:530


I feel like we should be able to use PipeBuffer/IOBuffer with pipeline.  Do 
the Julia devs believe we should never expect to be able to pipeline using 
IOBuffer/PipeBuffer?

It seems like many other people are expecting to be able to do this too:
https://github.com/JuliaLang/julia/issues/14437
https://github.com/JuliaLang/julia/issues/3823#issuecomment-157714083

A point brought up in the the issues say that one is a file abstraction and 
one is a stream abstraction.  I now have a bit better understanding of what 
that means but I couldn't find any official documentation on the 
differences between a stream and a file.
This is especially confusing if you consider 
filestream = open("somefile", "w")  - is a file as a stream abstraction but
buffer = IOBuffer() - is a stream (as the docs currently say) as a file 
abstraction.

The former works with pipeline, but not the latter.

In either case, it seems that I need an IOStream to pipe data into a 
pipeline command. How can I turn data into a stream that is not a file?
https://groups.google.com/forum/#!msg/julia-users/R-F3F97leh4/o4zKINZbbvUJ 
asks a similar question but readall produces a string, I still need to 
stream it into a pipeline command.

Thank you,
Will