[
https://issues.apache.org/jira/browse/BIGTOP-1680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14326326#comment-14326326
]
Konstantin Boudnik commented on BIGTOP-1680:
--------------------------------------------
Yes, I am aware that Spark is written in Scala. But so far it was contained to
the package creation and Bigtop itself didn't rely on Scala for the purpose of
the stack development.
I am not saying we should stay away from making Scala a requirement for the
dev. env (but I am certainly thinking it outloud :), but let's make this
decision with clear understanding of consequences not just implicitly by adding
a unit test elsewhere.
> [BigPetStore] BPS-Spark Unit tests for transaction generation
> -------------------------------------------------------------
>
> Key: BIGTOP-1680
> URL: https://issues.apache.org/jira/browse/BIGTOP-1680
> Project: Bigtop
> Issue Type: Test
> Components: blueprints
> Affects Versions: backlog
> Reporter: jay vyas
> Assignee: RJ Nowling
> Fix For: backlog
>
>
> The following code could be split up into a method, i think, and would be
> greatly served by a companion unit test.
> Its the reference impl of the data generator on spark: i.e. this is where we
> magically generate all the customer transactions .
> {noformat}
> 155 val transactionRDD = customerRDD.mapPartitionsWithIndex{
> 156 (index, custIter) =>
> 157 // Create a new RNG
> 158 val seedFactory = new SeedFactory(nextSeed ^ index)
> 159 val transactionIter = custIter.map{
> 160 customer =>
> 161 val products = productBC.value
> 162 //Create a new purchasing profile.
> 163 val profileGen = new PurchasingProfileGenerator(products,
> seedFactory)
> 164 val profile = profileGen.generate()
> 165 val transGen = new TransactionGenerator(customer, profile,
> storesBC.value, products, seedFact ory)
> 166 var transactions : List[Transaction] = List()
> 167 var transaction = transGen.generate()
> 168
> 169 //Create a list of this customer's transactions for the time
> period
> 170 while(transaction.getDateTime() < simulationLength) {
> 171 transactions = transaction :: transactions
> 172 transaction = transGen.generate()
> 173 }
> 174 //The final result, we return the list of transactions produced
> above.
> 175 transactions
> 176 }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
