Hi,
I have been running through some troubles while converting the code to Java.
I have done the matrix operations as directed and tried to find the maximum
score for each category. But the predicted category is mostly different from
the prediction done by MLlib.
I am fetching iterators of the pi, theta and testData to do my calculations.
pi and theta are in log space while my testData vector is not, could that
be a problem because I didn't see explicit conversion in Mllib also?
For example, for two categories and 5 features, I am doing the following
operation,
[1,2] + [1 2 3 4 5 ] * [1,2,3,4,5]
[6 7 8 9 10]
These are simple element-wise matrix multiplication and addition operators.
Following is the code,
Iterator<Tuple2<Object, Object>> piIterator =
piValue.iterator();
Iterator<Tuple2<Tuple2<Object, Object>, Object>>
thetaIterator = thetaValue.iterator();
Iterator<Tuple2<Object, Object>> testDataIterator = null;
double[] scores = new double[piValue.size()];
while (piIterator.hasNext()) {
double score = 0.0;
// reset to index 0
testDataIterator = testData.toBreeze().iterator();
while (testDataIterator.hasNext()) {
Tuple2<Object, Object> testTuple =
testDataIterator.next();
Tuple2<Tuple2<Object, Object>, Object> thetaTuple =
thetaIterator.next();
score += ((double) testTuple2._2 * (double)
thetaTuple2._2);
}
Tuple2<Object, Object> piTuple = piIterator.next();
score += (double) piTuple._2;
scores[(int) piTuple._1] = score;
if (maxScore < score) {
predictedCategory = (int) piTuple._1;
maxScore = score;
}
}
Where am I going wrong?
Thanks,
Jatin
-----
Novice Big Data Programmer
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Accessing-posterior-probability-of-Naive-Baye-s-prediction-tp19828p19968.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]