training_test condition never true for Multi-objective optimization

[zahidirfan]

PonyGE2/src/stats/stats.py

Line 243 in f337171

if hasattr(params['FITNESS_FUNCTION'], "training_test") and end:

The method of defining fitness functions in parameters is given in the documentation.
FITNESS_FUNCTION: [[FITNESS_FUNCTION_1_NAME], [FITNESS_FUNCTION_2_NAME]]

This results in an object : <class 'fitness.base_ff_classes.moo_ff.moo_ff'> which does not have an attribute 'training_test'.
AttributeError("'moo_ff' object has no attribute 'training_test'").

Therefore the test fitness and training fitness statistics are never generated for the multi-objective optimization

Since the training_test attribute in the fitness functions gets its value from params['DATASET_TEST']. The condition could be replaced to check if the paramater 'DATASET_TEST' has a value. However the

PonyGE2/src/stats/stats.py

Line 252 in f337171

ind.test_fitness = params['FITNESS_FUNCTION'](ind, dist='test')

Here we get the following error : TypeError("call() got an unexpected keyword argument 'dist'"), because the params['FITNESS'] is now an object moo_ff.

This could be resolved by updating the code with the following.

if params["DATASET_TEST"] and end:

        for ind in trackers.best_ever:
            # Iterate over all individuals in the first front.

            # Save training fitness.
            ind.training_fitness = copy(ind.fitness)

            # Evaluate test fitness.
            ind.test_fitness = [func(ind, dist="test") for func in params["FITNESS_FUNCTION"].fitness_functions]

            # Set main fitness as training fitness.
            ind.fitness = ind.training_fitness

[jmmcd]

I think we are agreeing we should get rid of the training_test attribute everywhere, right?

@zahidirfan could you please update the PR to make the same change in 1 more place in stats.py, and also to remove the 1 line in supervised_learning.py and regression_random_polynomial.py? I think it would be best to have all the changes in a single PR from yourself.

[zahidirfan]

@jmmcd: I have deleted the training_test attribute from the repository and added the check for DATASET_TEST to determine if the training and test datasets are included. I have checked the working by running the example using the following.

python ponyge.py --generations 10 --population 10 --fitness supervised_learning.regression_random_polynomial --extra_parameters 5 1 20 --grammar supervised_learning/supervised_learning.bnf --dataset_test Dummy

I was getting the following error and changed the grammar file.

"PonyGE2/src/fitness/supervised_learning/supervised_learning.py", line 111, in evaluate raise ValueError(shape_mismatch_txt) ValueError: Shape mismatch between y and yhat. Please check that your grammar uses the x[:, 0]style, notx[0]. Please see change at https://github.com/PonyGE/PonyGE2/issues/130.

I fixed it by changing the grammar file. I have also added the changed grammar file as a separate commit so that if it is decided not to change it, the change can be easily ignored.