Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

training_test condition never true for Multi-objective optimization #166

Open
zahidirfan opened this issue Oct 1, 2024 · 2 comments · May be fixed by #167
Open

training_test condition never true for Multi-objective optimization #166

zahidirfan opened this issue Oct 1, 2024 · 2 comments · May be fixed by #167

Comments

@zahidirfan
Copy link

zahidirfan commented Oct 1, 2024 •
edited
Loading

PonyGE2/src/stats/stats.py

Line 243 in f337171

if hasattr(params['FITNESS_FUNCTION'], "training_test") and end:

The method of defining fitness functions in parameters is given in the documentation.
FITNESS_FUNCTION: [[FITNESS_FUNCTION_1_NAME], [FITNESS_FUNCTION_2_NAME]]

This results in an object : <class 'fitness.base_ff_classes.moo_ff.moo_ff'> which does not have an attribute 'training_test'.
AttributeError("'moo_ff' object has no attribute 'training_test'").

Therefore the test fitness and training fitness statistics are never generated for the multi-objective optimization

Since the training_test attribute in the fitness functions gets its value from params['DATASET_TEST']. The condition could be replaced to check if the paramater 'DATASET_TEST' has a value. However the

PonyGE2/src/stats/stats.py

Line 252 in f337171

ind.test_fitness = params['FITNESS_FUNCTION'](ind, dist='test')

Here we get the following error : TypeError("call() got an unexpected keyword argument 'dist'"), because the params['FITNESS'] is now an object moo_ff.

This could be resolved by updating the code with the following.

if params["DATASET_TEST"] and end:

        for ind in trackers.best_ever:
            # Iterate over all individuals in the first front.

            # Save training fitness.
            ind.training_fitness = copy(ind.fitness)

            # Evaluate test fitness.
            ind.test_fitness = [func(ind, dist="test") for func in params["FITNESS_FUNCTION"].fitness_functions]

            # Set main fitness as training fitness.
            ind.fitness = ind.training_fitness
zahidirfan added a commit to zahidirfan/PonyGE2 that referenced this issue Oct 2, 2024
@zahidirfan
Fix to get Test and Train Fitness for final solution of Mult-objectiv…
4e1b5da 
…e Optimization. Fixes PonyGE#166
@zahidirfan zahidirfan linked a pull request Oct 2, 2024 that will close this issue
Open
zahidirfan added a commit to zahidirfan/PonyGE2 that referenced this issue Oct 2, 2024
@zahidirfan
In addition to the fix of training_test condition never true for Mult…
471ebe7 
…i-objective optimization PonyGE#166. This was fixed but not pushed in original commit.
@jmmcd
Copy link
Collaborator

jmmcd commented Oct 10, 2024

I think we are agreeing we should get rid of the training_test attribute everywhere, right?

@zahidirfan could you please update the PR to make the same change in 1 more place in stats.py, and also to remove the 1 line in supervised_learning.py and regression_random_polynomial.py? I think it would be best to have all the changes in a single PR from yourself.

@zahidirfan
Copy link
Author

@jmmcd: I have deleted the training_test attribute from the repository and added the check for DATASET_TEST to determine if the training and test datasets are included. I have checked the working by running the example using the following.

python ponyge.py --generations 10 --population 10 --fitness supervised_learning.regression_random_polynomial --extra_parameters 5 1 20 --grammar supervised_learning/supervised_learning.bnf --dataset_test Dummy

I was getting the following error and changed the grammar file.

"PonyGE2/src/fitness/supervised_learning/supervised_learning.py", line 111, in evaluate raise ValueError(shape_mismatch_txt) ValueError: Shape mismatch between y and yhat. Please check that your grammar uses the x[:, 0]style, notx[0]. Please see change at https://github.com/PonyGE/PonyGE2/issues/130.

I fixed it by changing the grammar file. I have also added the changed grammar file as a separate commit so that if it is decided not to change it, the change can be easily ignored.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
2 participants
@jmmcd @zahidirfan