Hey Mich
glad to know u got to the bottom
In python, if you want to run a module - same as if you would use
Java/Scala -you will have to define a def main() method
You'll notice that the snippet i sent you had this syntax -
if __name__ == "main":
main()
I am guessing you just choose an
Thanks all.
Found out the problem :(
I defined the runner.py as
class main()
I replaced it with
def main():
and it worked without declaring numRows as global.
I am still wondering the reason for it working with def main()?
regards,
Mich
*Disclaimer:* Use it at your own risk. Any and
Hi All,
I am using Standalone Spark.
I am using dynamic memory allocation. Despite giving max executors, min
executors and initial executors, my streaming job is taking all executors
available in the cluster. Could anyone please suggest what can be wrong
here?
Please note source is Kafka.
I
Hi All,
Any help in writing a code to find longest Continuous Subsequence between
two columns
Like :COL1:"sparkJava' col2:Java8 -->Result :Java
Thanks in advance
Regards
Raviteja
I don't believe you'll be able to use globals in a Spark task, as they
won't exist on the remote executor machines.
On Sun, Dec 13, 2020 at 3:46 AM Mich Talebzadeh
wrote:
> thanks Marco.
>
> When I stripped down spark etc and ran your map, it came back OK (no
> errors) WITHOUT global numRows
>
Sure Mich...uhm...let me try to run your code in my IDE. .. I m intrigued
by the error..
Will report back either if I find something or not.
Kind regards
On Sun, Dec 13, 2020, 9:46 AM Mich Talebzadeh
wrote:
> thanks Marco.
>
> When I stripped down spark etc and ran your map, it came back OK (no
