Adding generic methods to switch models from Float32 to Float64 and other precisions

[AntoninKns]

I am currently working on a Levenberg-Marquardt algorithm and I am using NLPModels to solve it.

My model has the following form :

BundleAdjustmentModel{Float32,Vector{Float32}} <: AbstractNLSModel{Float32,Vector{Float32}}

and I want to do the following code.

levenberg_marquardt(model)
convert_type(Float64,model)
levenberg_marquardt(model)

The idea is to start solving the problem using Float32 up until certain criterias are met, then switch the model to Float64 adn then solve the problem from this point but using Float64.

I created a method convert_type(T::Type, model::BundleAdjustmentModel) that copies every element from my model from the current precision to a new one and keeps counters and the rest of the meta-model unchanged.

Currently there is no such generic method in NLPModels. I was discussing with @tmigot about implementing one in order to have a nice generic way to do it across all the different NLPModels subtypes the same way it works for other functions in the API for example the obj function.

Following @tmigot idea we might instead add a way to create a model in every necessary precision beforehand and add a function to switch from one precision to the other without needing further allocations.

The idea is to have something that kind of looks like this :

mutable struct MultiPrecisionNLPModel{Float,Vector} <: AbstractNLSModel{T,S}
  NLPModel_Float32{Float32,Vector{Float32}}
  NLPModel_Float64{Float64,Vector{Float64}}
  NLPModel_Float128{Float128,Vector{Float128}}
end

and then having a function switch( :: Type, :: AbstractNLSModel) that allows to go from one precision to the other.

Some questions arise from this :

1. Do we keep a single meta model across all precisions ?
2. Do we create a meta model for the bigger structure and then another one for all the precisions ?
3. Same question for the counters ?
4. Do we create a new structure that encompass the current AbstractNLPModel type or do we create a subtype that allows multiple precisions ?

[AntoninKns]

@dpo @tmigot @amontoison

[dpo]

I would suggest having multiple solvers instead of multiple models. If your model is written sufficiently generically (as are the BundleAdjustmentModels), you simply evaluate one of the problem functions at an x in the floating-point system you want, and receive a result in the same floating-point system. However, it's the solver object that preallocates work space for the solve. You could even imagine having a solver object that preallocates storage in several floating-point systems at once.