Re: env.fromElements produces TypeInformation error
Hi,
I think the problem is that the case class has generic parameters. You can
try making TypeInformation for those parameters implicitly available at the
call site, i.e:
implicit val typeT = createTypeInformation[T] // where you insert the
specific type for T and do the same for the other generic parameters
Maybe this helps.
Cheers,
Aljoscha
On Sat, 4 Jun 2016 at 20:04 Dan Drewes wrote:
> I've tested it, but unfortunately it does not solve the problem.
> The error message remains the same.
>
>
> Am 04.06.2016 um 19:38 schrieb Simone Robutti:
>
> I'm not sure if this is the solution and I don't have the possibility to
> try right now, but you should move the case class "State" definition
> outside the abstract class.
>
> 2016-06-04 17:34 GMT+02:00 Dan Drewes :
>
>>
>> Hi,
>>
>> compiling the code:
>>
>> def minimize(f: DF, init: T): T = {
>>
>> //create execution environment val env =
>> ExecutionEnvironment.getExecutionEnvironment
>>
>> val initialstate = initialState(f, init)
>> val iterativestate= env.fromElements(initialstate).iterate(1) {
>> iterationInput: DataSet[State] =>
>> val result = iterationInput.map {
>> oldstate => computeNextState(adjustedFun, oldstate)
>> }
>> result
>> }
>> }
>>
>> object IterationsFirstOrderMinimizer { case class State[+T,
>> +ConvergenceInfo, +History] (x: T,
>>value: Double, grad: T,
>>adjustedValue: Double,
>> adjustedGradient: T,
>>iter: Int,
>>initialAdjVal: Double,
>>history: History,
>>convergenceInfo:
>> ConvergenceInfo,
>>searchFailed: Boolean =
>> false) {
>> }
>>
>> ... fails with the error: could not find implicit value for evidence
>> parameter of type
>> org.apache.flink.api.common.typeinfo.TypeInformation[IterationsFirstOrderMinimizer.this.State]
>> val iterativestate= env.fromElements(initialState(f,
>> init)).iterate(1) { ^ Google
>> only told me to import org.apache.flink.api.scala._ which i do. The
>> other suggested solution for generic methods (
>> https://ci.apache.org/projects/flink/flink-docs-master/internals/types_serialization.html)
>> does also not apply because it's the fromElements call that produces the
>> error. I am very new to Flink and Scala and because I don't know if the
>> code I posted above is enough to say what I'm doing wrong, you can find the
>> complete scala file attached. Thanks in advance for any ideas and hints.
>> Best, Dan
>>
>>
>>
>> Virenfrei.
>> www.avast.com
>>
>>
>
>
>
>
>
> Virenfrei.
> www.avast.com
>
>
Re: env.fromElements produces TypeInformation error
I've tested it, but unfortunately it does not solve the problem.
The error message remains the same.
Am 04.06.2016 um 19:38 schrieb Simone Robutti:
I'm not sure if this is the solution and I don't have the possibility
to try right now, but you should move the case class "State"
definition outside the abstract class.
2016-06-04 17:34 GMT+02:00 Dan Drewes mailto:dre...@campus.tu-berlin.de>>:
Hi,
compiling the code:
def minimize(f:DF, init:T):T = {
//create execution environment val env =
ExecutionEnvironment.getExecutionEnvironment
valinitialstate =initialState(f, init)
val iterativestate= env.fromElements(initialstate).iterate(1) {
iterationInput: DataSet[State] =>
val result = iterationInput.map {
oldstate => computeNextState(adjustedFun, oldstate)
}
result
}
}
objectIterationsFirstOrderMinimizer {
case class State[+T,+ConvergenceInfo,+History] (x:T,
value: Double, grad:T,
adjustedValue: Double,
adjustedGradient:T,
iter: Int,
initialAdjVal: Double,
history:History,
convergenceInfo:ConvergenceInfo,
searchFailed: Boolean
=false) {
}
... fails with the error: could not find implicit value for
evidence parameter of type
org.apache.flink.api.common.typeinfo.TypeInformation[IterationsFirstOrderMinimizer.this.State]
val iterativestate= env.fromElements(initialState(f,
init)).iterate(1) { ^
Google only told me toimport org.apache.flink.api.scala._
which i do. The other suggested solution for generic methods
(https://ci.apache.org/projects/flink/flink-docs-master/internals/types_serialization.html)
does also not apply because it's the fromElements call that
produces the error. I am very new to Flink and Scala and because I
don't know if the code I posted above is enough to say what I'm
doing wrong, you can find the complete scala file attached. Thanks
in advance for any ideas and hints. Best, Dan
Virenfrei. www.avast.com
---
Diese E-Mail wurde von Avast Antivirus-Software auf Viren geprüft.
https://www.avast.com/antivirus
Re: env.fromElements produces TypeInformation error
I'm not sure if this is the solution and I don't have the possibility to
try right now, but you should move the case class "State" definition
outside the abstract class.
2016-06-04 17:34 GMT+02:00 Dan Drewes :
>
> Hi,
>
> compiling the code:
>
> def minimize(f: DF, init: T): T = {
>
> //create execution environment val env =
> ExecutionEnvironment.getExecutionEnvironment
>
> val initialstate = initialState(f, init)
> val iterativestate= env.fromElements(initialstate).iterate(1) {
> iterationInput: DataSet[State] =>
> val result = iterationInput.map {
> oldstate => computeNextState(adjustedFun, oldstate)
> }
> result
> }
> }
>
> object IterationsFirstOrderMinimizer { case class State[+T,
> +ConvergenceInfo, +History] (x: T,
>value: Double, grad: T,
>adjustedValue: Double,
> adjustedGradient: T,
>iter: Int,
>initialAdjVal: Double,
>history: History,
>convergenceInfo:
> ConvergenceInfo,
>searchFailed: Boolean =
> false) {
> }
>
> ... fails with the error: could not find implicit value for evidence
> parameter of type
> org.apache.flink.api.common.typeinfo.TypeInformation[IterationsFirstOrderMinimizer.this.State]
> val iterativestate= env.fromElements(initialState(f,
> init)).iterate(1) { ^ Google
> only told me to import org.apache.flink.api.scala._ which i do. The
> other suggested solution for generic methods (
> https://ci.apache.org/projects/flink/flink-docs-master/internals/types_serialization.html)
> does also not apply because it's the fromElements call that produces the
> error. I am very new to Flink and Scala and because I don't know if the
> code I posted above is enough to say what I'm doing wrong, you can find the
> complete scala file attached. Thanks in advance for any ideas and hints.
> Best, Dan
>
>
>
> Virenfrei.
> www.avast.com
>
>
env.fromElements produces TypeInformation error
Hi,
compiling the code:
def minimize(f:DF, init:T):T = {
//create execution environment val env =
ExecutionEnvironment.getExecutionEnvironment
valinitialstate =initialState(f, init)
val iterativestate= env.fromElements(initialstate).iterate(1) {
iterationInput: DataSet[State] =>
val result = iterationInput.map {
oldstate => computeNextState(adjustedFun, oldstate)
}
result
}
}
objectIterationsFirstOrderMinimizer {
case class State[+T,+ConvergenceInfo,+History] (x:T,
value: Double, grad:T,
adjustedValue: Double,
adjustedGradient:T,
iter: Int,
initialAdjVal: Double,
history:History,
convergenceInfo:ConvergenceInfo,
searchFailed: Boolean
=false) {
}
... fails with the error: could not find implicit value for evidence
parameter of type
org.apache.flink.api.common.typeinfo.TypeInformation[IterationsFirstOrderMinimizer.this.State]
val iterativestate= env.fromElements(initialState(f,
init)).iterate(1) { ^ Google
only told me toimport org.apache.flink.api.scala._ which i do. The
other suggested solution for generic methods
(https://ci.apache.org/projects/flink/flink-docs-master/internals/types_serialization.html)
does also not apply because it's the fromElements call that produces the
error. I am very new to Flink and Scala and because I don't know if the
code I posted above is enough to say what I'm doing wrong, you can find
the complete scala file attached. Thanks in advance for any ideas and
hints. Best, Dan
---
Diese E-Mail wurde von Avast Antivirus-Software auf Viren geprüft.
https://www.avast.com/antivirus
package breeze.optimize
import breeze.linalg.norm
import breeze.math.{MutableEnumeratedCoordinateField,
MutableFiniteCoordinateField, NormedModule}
import breeze.optimize.IterationsFirstOrderMinimizer.ConvergenceReason
import breeze.stats.distributions.{RandBasis, ThreadLocalRandomGenerator}
import breeze.util.Implicits._
import breeze.util.SerializableLogging
import org.apache.commons.math3.random.MersenneTwister
import IterationsFirstOrderMinimizer.ConvergenceCheck
import org.apache.flink.api.common.ExecutionConfig
import org.apache.flink.api.common.typeinfo.TypeInformation
import org.apache.flink.api.common.typeutils.TypeSerializer
import org.apache.flink.api.scala.DataSet
import org.apache.flink.api.scala.ExecutionEnvironment
import scala.reflect._
import scala.reflect.runtime.universe._
import org.apache.flink.api.common.functions.MapFunction
import org.apache.flink.api.common.functions.ReduceFunction
import org.apache.flink.api.common.functions.RichMapFunction
import org.apache.flink.api.scala._
/**
*
* @author
*/
abstract class IterationsFirstOrderMinimizer[T,
DF<:StochasticDiffFunction[T]](val convergenceCheck: ConvergenceCheck[T])
(implicit space: NormedModule[T, Double]) extends Minimizer[T,DF] with
SerializableLogging {
def this(maxIter: Int = -1,
tolerance: Double = 1E-6,
fvalMemory: Int = 100,
relativeTolerance: Boolean = true)(implicit space: NormedModule[T,
Double]) =
this(IterationsFirstOrderMinimizer.defaultConvergenceCheck[T](maxIter,
tolerance, relativeTolerance, fvalMemory))
/**
* Any history the derived minimization function needs to do its updates.
typically an approximation
* to the second derivative/hessian matrix.
*/
type History
type State = IterationsFirstOrderMinimizer.State[T, convergenceCheck.Info,
History]
import space.normImpl
protected def initialHistory(f: DF, init: T): History
protected def adjustFunction(f: DF): DF = f
protected def adjust(newX: T, newGrad: T, newVal: Double):(Double,T) =
(newVal,newGrad)
protected def chooseDescentDirection(state: State, f: DF):T
protected def determineStepSize(state: State, f: DF, direction: T):Double
protected def takeStep(state: State, dir: T, stepSize:Double):T
protected def updateHistory(newX: T, newGrad: T, newVal: Double, f: DF,
oldState: State):History
protected def initialState(f: DF, init: T): State = {
val x = init
val history = initialHistory(f,init)
val (value, grad) = calculateObjective(f, x, history)
val (adjValue,adjGrad) = adjust(x,grad,value)
IterationsFirstOrderMinimizer.State(x,value,grad,adjValue,adjGrad,0,adjValue,history,
convergenceCheck.initialInfo)
}
protected def calculateObjective(f: DF, x: T, history: History): (Double, T)
= {
f.calculate(x)
}
def infiniteIterations(f: DF, state: State): Iterator[State] = {
var failedOnce = false
val adj
