As suggested, here is a working solution using a Role::Tiny role to modify/override Mojo::Parameters::to_string so that it does not escape the caret '^' character. Feel free to comment/criticize.
Thank you all for your valuable input, this was enlightening. use Mojo::UserAgent; use FindBin qw($Bin); use feature qw(say); use lib $Bin; $| = 1; # Accept responses of indefinite size my $ua = Mojo::UserAgent->new(max_response_size => 0); $ua->inactivity_timeout(0); my $url_raw = 'https://streamerapi.finance.yahoo.com/streamer/1.0?s= ^GSPC&k=l86,l84,p20&callback=parent.yfs_u1f&mktmcb=parent.yfs_mktmcb&gencallback=parent.yfs_gencb&mu=1&lang=en-US®ion=US&localize=0' ; # Build a normal transaction my $tx = $ua->build_tx(GET => $url_raw, => {Accept => '*/*'}); # Bypass caret '^' encoding with a Role::Tiny role as Yahoo finance servers are not RFC 3986 compliant yet... $tx->req->url->query->with_roles('+Unescaped'); say $tx->req->to_string; # Replace "read" events to disable default content parser $tx->res->content->unsubscribe('read')->on( read => sub { my ($content, $bytes) = @_; say "Streaming: $bytes"; } ); # Process transaction $tx = $ua->start($tx); say "done"; And the file ./Mojo/Parameters/Role/Unescaped.pm to modify/override Mojo::Parameters::to_string package Mojo::Parameters::Role::Unescaped; use Mojo::Base -role; use Mojo::Util qw(decode encode url_escape url_unescape); has charset => 'UTF-8'; # Override default Mojo::Parameters::to_string method sub to_string { my $self = shift; # String (RFC 3986) my $charset = $self->charset; if (defined(my $str = $self->{string})) { $str = encode $charset, $str if $charset; # Do not encode caret '^' character return url_escape $str, '^A-Za-z0-9\-._~%!$&\'()*+,;=:@/?i^'; } } 1; On Saturday, October 27, 2018 at 5:38:43 PM UTC-6, Sylvain Thibault wrote: > > Given URL https://somehost.com.com/streamer/1.0?s=^GSPC > > When doing a GET, Mojo::UserAgent encodes the caret '^' as %5E > The server returns NOT FOUND. > > So this transaction from Mojo::UserAgent returns NOT FOUND: > > GET /streamer/1.0?s=%5EGSPC&k=l86,l84,p20 HTTP/1.1 > Host: somehost.com > Accept: */* > User-Agent: Mojolicious (Perl) > Content-Length: 0 > > This transaction using curl with the caret not encoded returns the desired > output: > > GET /streamer/1.0?s=^GSPC&k=l86,l84,p20 HTTP/1.1 > Host: s <http://streamerapi.finance.yahoo.com/>omehost.com > User-Agent: curl/7.61.1 > Accept: */* > > How does one send a caret '^' in a URL without encoding it to %5E ? > > Using a symbol without a caret works great in the Mojo::UserAgent version > of the code. > > Thanks, > > Sylvain Thibault > On Saturday, October 27, 2018 at 5:38:43 PM UTC-6, Sylvain Thibault wrote: > > Given URL https://somehost.com.com/streamer/1.0?s=^GSPC > > When doing a GET, Mojo::UserAgent encodes the caret '^' as %5E > The server returns NOT FOUND. > > So this transaction from Mojo::UserAgent returns NOT FOUND: > > GET /streamer/1.0?s=%5EGSPC&k=l86,l84,p20 HTTP/1.1 > Host: somehost.com > Accept: */* > User-Agent: Mojolicious (Perl) > Content-Length: 0 > > This transaction using curl with the caret not encoded returns the desired > output: > > GET /streamer/1.0?s=^GSPC&k=l86,l84,p20 HTTP/1.1 > Host: s <http://streamerapi.finance.yahoo.com/>omehost.com > User-Agent: curl/7.61.1 > Accept: */* > > How does one send a caret '^' in a URL without encoding it to %5E ? > > Using a symbol without a caret works great in the Mojo::UserAgent version > of the code. > > Thanks, > > Sylvain Thibault > > -- You received this message because you are subscribed to the Google Groups "Mojolicious" group. To unsubscribe from this group and stop receiving emails from it, send an email to mojolicious+unsubscr...@googlegroups.com. To post to this group, send email to mojolicious@googlegroups.com. Visit this group at https://groups.google.com/group/mojolicious. For more options, visit https://groups.google.com/d/optout.
