Hi,
I am trying to submit a custom metrics.properties file to enable the
collection of spark metrics, but I am having a hard time even starting it
in local mode.
spark-submit \
...
--files "./metrics.properties"
--conf "spark.metrics.conf=metrics.properties"
...
However I am
up to 3:00am, v2 after that). Manual state management is also
> supported by the framework but it’s harder to control because:
>
>- you’re not guaranteed to shut down gracefully
>- You may have a bug that prevents the state to be saved and you can’t
>restart the app w/o up
this situation too ?
On Fri, Sep 25, 2015 at 12:20 PM, Cody Koeninger <c...@koeninger.org> wrote:
> Storing passbacks transactionally with results in your own data store,
> with a schema that makes sense for you, is the optimal solution.
>
> On Fri, Sep 25, 2015 at 11:05 AM,
Hi,
in my application I use Kafka direct streaming and I have also enabled
checkpointing.
This seems to work fine if the application is restarted. However if I
change the code and resubmit the application, it cannot start because of
the checkpointed data being of different class versions.
Is there
wrote:
> No, you cant use checkpointing across code changes. Either store offsets
> yourself, or start up your new app code and let it catch up before killing
> the old one.
>
> On Thu, Sep 24, 2015 at 8:40 AM, Radu Brumariu <bru...@gmail.com
> <javascript:_e(%7B%7D,'cvml
has been discussed numerous times, TD's response has consistently
> been that it's unlikely to be possible
>
> On Thu, Sep 24, 2015 at 12:26 PM, Radu Brumariu <bru...@gmail.com
> <javascript:_e(%7B%7D,'cvml','bru...@gmail.com');>> wrote:
>
>> It seems to me that this
