unrecognized arguments and info = coco['info'] #1

favervn · 2020-05-15T06:58:37Z

I encounter this issue, please help me to solve it:

- When I type the full command:
python cocosplit.py --having-annotaions -s 0.8 D:/IMG_from_VID/COCO_Datasets/data/trainval.json train.json test.json

It shows this mess:

usage: cocosplit.py [-h] -s SPLIT [--having-annotations]
coco_annotations train test

cocosplit.py: error: unrecognized arguments: --having-annotaions

- When I remove the argument "--having-annotaions", it shows:

Traceback (most recent call last):
File "cocosplit.py", line 52, in
main(args)
File "cocosplit.py", line 30, in main
info = coco['info']
KeyError: 'info'

Thanks

The text was updated successfully, but these errors were encountered:

nofficer · 2020-05-21T04:17:26Z

Your coco dataset probably doesn't have info. Remove all instances of info in cocosplit.py, you're trying to reference something that isn't in your initial JSON data.

morganaribeiro · 2020-12-09T15:41:01Z

@nofficer, @favervn and @akarazniewicz I ran coco_split via Colab for my custom dataset:

!git clone https://github.com/akarazniewicz/cocosplit.git
!ls
Output: cocosplit sample_data
%cd cocosplit/
Output: /content/cocosplit
!ls
Output: cocosplit.py README.md requirements.txt
!pip install -r requirements.txt
I connected to the drive to access the Data along with the json file.

from google.colab import drive
drive.mount('/content/drive')

There was a syntax error, but I generated a json file with my complete Dataset after making the annotations via Labelme.

python cocosplit.py --having-annotations -s 0.8 /content/drive/MyDrive/Dataset-COCO-Split/coco_annotations.json train.json test.json`

Error:

File "<ipython-input-10-a30723afe766>", line 1
    python cocosplit.py --having-annotations -s 0.8 /content/drive/MyDrive/Dataset-COCO-Split/coco_annotations.json train.json test.json
                   ^
SyntaxError: invalid syntax

@nofficer, @favervn and @akarazniewicz Would you help me?

minooei · 2020-12-14T14:06:49Z

@nofficer, @favervn and @akarazniewicz I ran coco_split via Colab for my custom dataset:

!git clone https://github.com/akarazniewicz/cocosplit.git

!ls
Output: cocosplit sample_data

%cd cocosplit/
Output: /content/cocosplit

!ls
Output: cocosplit.py README.md requirements.txt

!pip install -r requirements.txt

I connected to the drive to access the Data along with the json file.
from google.colab import drive
drive.mount('/content/drive') 
There was a syntax error, but I generated a json file with my complete Dataset after making the annotations via Labelme.
python cocosplit.py --having-annotations -s 0.8 /content/drive/MyDrive/Dataset-COCO-Split/coco_annotations.json train.json test.json`
Error:
File "<ipython-input-10-a30723afe766>", line 1
    python cocosplit.py --having-annotations -s 0.8 /content/drive/MyDrive/Dataset-COCO-Split/coco_annotations.json train.json test.json
                   ^
SyntaxError: invalid syntax
@nofficer, @favervn and @akarazniewicz Would you help me?

it seems you forgot to put an exclamation mark (!) before python command in google colab.
alternatively, you can copy-paste python codes from cocosplit.py in a colab cell.

morganaribeiro · 2020-12-14T15:01:46Z

@minooei I tested it by placing (!) Before the python command in Google Colab. However, this error appeared below:

Traceback (most recent call last):
  File "cocosplit.py", line 52, in <module>
    main(args)
  File "cocosplit.py", line 30, in main
    info = coco['info']
KeyError: 'info'

Any suggestions @minooei?

morganaribeiro · 2020-12-14T15:08:00Z

@minooei I tried the alternative of copying the code from the cocosplit.py file in a cell and the output was this:

usage: ipykernel_launcher.py [-h] -s SPLIT [--having-annotations]
                             coco_annotations train test
ipykernel_launcher.py: error: the following arguments are required: train, test, -s
An exception has occurred, use %tb to see the full traceback.

SystemExit: 2
/usr/local/lib/python3.6/dist-packages/IPython/core/interactiveshell.py:2890: UserWarning: To exit: use 'exit', 'quit', or Ctrl-D.
  warn("To exit: use 'exit', 'quit', or Ctrl-D.", stacklevel=1)

minooei · 2020-12-14T15:11:17Z

@minooei I tested it by placing (!) Before the python command in Google Colab. However, this error appeared below:
Traceback (most recent call last):
  File "cocosplit.py", line 52, in <module>
    main(args)
  File "cocosplit.py", line 30, in main
    info = coco['info']
KeyError: 'info'
Any suggestions @minooei?

just simply delete lines 30,31 and change line 20:

        json.dump({ 'images': images, 
            'annotations': annotations, 'categories': categories}, coco, indent=2, sort_keys=True)

you need some basic python knowledge.

morganaribeiro · 2020-12-14T18:02:53Z

@minooei In the case where there is the term (info) should I delete it?
Also in this line:

save_coco(args.train, info, licenses, x, filter_annotations(annotations, x), categories)
 save_coco(args.test, info, licenses, y, filter_annotations(annotations, y), categories)

minooei · 2020-12-14T18:13:51Z

@minooei In the case where there is the term (info) should I delete it?
Also in this line:
save_coco(args.train, info, licenses, x, filter_annotations(annotations, x), categories)
 save_coco(args.test, info, licenses, y, filter_annotations(annotations, y), categories)

yes, since your dataset does not have any info you can delete or just replace with empty dict : {}
save_coco(args.train, {}, {}, x, filter_annotations(annotations, x), categories)

morganaribeiro · 2020-12-14T19:38:43Z

@minooei Thanks, would you have any python courses that you could refer me to?

minooei · 2020-12-14T20:21:34Z

@minooei Thanks, would you have any python courses that you could refer me to?
see this list:
https://github.com/quobit/awesome-python-in-education#courses-and-lessons

morganaribeiro · 2021-01-18T20:55:26Z

@minooei Do you know if it is possible to change the ratio of the Script to 70 (training) / 15 (test) / and 15 (validation)?

minooei · 2021-01-18T21:42:22Z

@minooei Do you know if it is possible to change the ratio of the Script to 70 (training) / 15 (test) / and 15 (validation)?

you can split the test set in two half and use it for validation

        x, y = train_test_split(images, train_size=args.split)
        z, y = train_test_split(y, train_size=0.5)

        save_coco(args.train, info, licenses, x, filter_annotations(annotations, x), categories)
        save_coco(args.test, info, licenses, y, filter_annotations(annotations, y), categories)
        save_coco('val.json', info, licenses, z, filter_annotations(annotations, z), categories)

morganaribeiro · 2021-01-18T22:08:21Z

Can you explain to me how the ratio is by placing this line with train_size = 0.5, how much is left for the other train and test? z, y = train_test_split (y, train_size = 0.5) Em seg, 18 de jan de 2021 18:42, mohammad minooei <[email protected]> escreveu:

…

@minooei <https://github.com/minooei> Do you know if it is possible to change the ratio of the *Script* to 70 (training) / 15 (test) / and 15 (validation)? you can split the test set in two half and use it for validation x, y = train_test_split(images, train_size=args.split) z, y = train_test_split(y, train_size=0.5) save_coco(args.train, info, licenses, x, filter_annotations(annotations, x), categories) save_coco(args.test, info, licenses, y, filter_annotations(annotations, y), categories) save_coco('val.json', info, licenses, z, filter_annotations(annotations, z), categories) — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AK3B5AR6LYK5UIVCD7MV533S2STMXANCNFSM4NBKBJFQ> .

minooei · 2021-01-18T22:27:01Z

Can you explain to me how the ratio is by placing this line with train_size = 0.5, how much is left for the other train and test? z, y = train_test_split (y, train_size = 0.5) Em seg, 18 de jan de 2021 18:42, mohammad minooei [email protected] escreveu:
…
@minooei https://github.com/minooei Do you know if it is possible to change the ratio of the Script to 70 (training) / 15 (test) / and 15 (validation)? you can split the test set in two half and use it for validation x, y = train_test_split(images, train_size=args.split) z, y = train_test_split(y, train_size=0.5) save_coco(args.train, info, licenses, x, filter_annotations(annotations, x), categories) save_coco(args.test, info, licenses, y, filter_annotations(annotations, y), categories) save_coco('val.json', info, licenses, z, filter_annotations(annotations, z), categories) — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AK3B5AR6LYK5UIVCD7MV533S2STMXANCNFSM4NBKBJFQ .

don't let the names of the function and argument makes you confused. train_test_split is just a function that gets an array and split it by a ratio. you can call this function multiple times without affecting previous splits.
https://datascience.stackexchange.com/questions/15135/train-test-validation-set-splitting-in-sklearn

morganaribeiro · 2021-01-18T22:44:26Z

But the second line, train_size = 0.1, is the percentage for Validation only in this case? Because if I pass as parameter in the line of terminal 0.7 it is for train.json and 0.2 for test.json. Closing 100%. Is the logic right? Em seg, 18 de jan de 2021 19:27, mohammad minooei <[email protected]> escreveu:

…

Can you explain to me how the ratio is by placing this line with train_size = 0.5, how much is left for the other train and test? z, y = train_test_split (y, train_size = 0.5) Em seg, 18 de jan de 2021 18:42, mohammad minooei ***@***.*** escreveu: … <#m_-5322528320105522290_> @minooei <https://github.com/minooei> https://github.com/minooei Do you know if it is possible to change the ratio of the *Script* to 70 (training) / 15 (test) / and 15 (validation)? you can split the test set in two half and use it for validation x, y = train_test_split(images, train_size=args.split) z, y = train_test_split(y, train_size=0.5) save_coco(args.train, info, licenses, x, filter_annotations(annotations, x), categories) save_coco(args.test, info, licenses, y, filter_annotations(annotations, y), categories) save_coco('val.json', info, licenses, z, filter_annotations(annotations, z), categories) — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1 (comment) <#1 (comment)>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AK3B5AR6LYK5UIVCD7MV533S2STMXANCNFSM4NBKBJFQ . don't let the names of the function and argument makes you confused. train_test_split is just a function that gets an array and split it by a ratio. you can call this function multiple times without affecting previous splits. https://datascience.stackexchange.com/questions/15135/train-test-validation-set-splitting-in-sklearn — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AK3B5ATCDNCNVPVEQV2CWTDS2SYUJANCNFSM4NBKBJFQ> .

morganaribeiro · 2021-01-19T00:25:10Z

@minooei Thanks for the explanation.

(1) I formatted the code like this:

import json
import argparse
import funcy
from sklearn.model_selection import train_test_split

parser = argparse.ArgumentParser(description='Splits COCO annotations file into training, test and validation sets.')
parser.add_argument('annotations', metavar='coco_annotations', type=str,
                    help='Path to COCO annotations file.')
parser.add_argument('train', type=str, help='Where to store COCO training annotations')
parser.add_argument('test', type=str, help='Where to store COCO test annotations')
parser.add_argument('val', type=str, help='Where to store COCO val annotations')
parser.add_argument('-s', dest='split', type=float, required=True,
                    help="A percentage of a split; a number in (0, 1)")
parser.add_argument('--having-annotations', dest='having_annotations', action='store_true',
                    help='Ignore all images without annotations. Keep only these with at least one annotation')


args = parser.parse_args()

def save_coco(file, info, licenses, images, annotations, categories):
    with open(file, 'wt', encoding='UTF-8') as coco:
        json.dump({ 'info': info, 'licenses': licenses, 'images': images, 
            'annotations': annotations, 'categories': categories}, coco, indent=2, sort_keys=True)

def filter_annotations(annotations, images):
    image_ids = funcy.lmap(lambda i: int(i['id']), images)
    return funcy.lfilter(lambda a: int(a['image_id']) in image_ids, annotations)

def main(args):
    with open(args.annotations, 'rt', encoding='UTF-8') as annotations:
        coco = json.load(annotations)
        info = coco['info']
        licenses = coco['licenses']
        images = coco['images']
        annotations = coco['annotations']
        categories = coco['categories']

        number_of_images = len(images)

        images_with_annotations = funcy.lmap(lambda a: int(a['image_id']), annotations)

        if args.having_annotations:
            images = funcy.lremove(lambda i: i['id'] not in images_with_annotations, images)

        train_X, test_Y = train_test_split(images, train_size=args.split) # 60% train - 30% teste
        train_X, val_Z = train_test_split(test_Y, train_size=0.50)  # 30%/2 = 0.15 e 0.15

        save_coco(args.train, info, licenses, train_X, filter_annotations(annotations, train_X), categories)
        save_coco(args.test, info, licenses, test_Y, filter_annotations(annotations, test_Y), categories)
        save_coco(args.val, info, licenses, val_Z, filter_annotations(annotations, val_Z), categories)

        print("Saved {} entries in {} and {} in {}".format(len(train_X), args.train, len(test_Y), args.test, len(val_Z), args.val))
        print("Number of images in the train set:", len(train_X))
        print("Number of images in the test set:", len(test_Y))
        print("Number of images in the validation set:", len(val_Z))
        

if __name__ == "__main__":
    main(args)

(2) Then at the terminal I tried to execute the following command:
python coco_split_train_test_val.py -s 0.6 coco_annotations.json train.json test.json val.json

(3) However, the following error occurred, would you have any suggestions:

Dataset-COCO-Split>python coco_split_train_test_val.py -s 0.6 coco_annotations.json train.json test.json val.json
Traceback (most recent call last):
  File "coco_split_train_test_val.py", line 3, in <module>
    import funcy
ModuleNotFoundError: No module named 'funcy'

Would you help me?

minooei · 2021-01-19T08:16:11Z

    train_X, val_Z = train_test_split(test_Y, train_size=0.50)  # 30%/2 = 0.15 e 0.15

simply install funcy by: pip install funcy

and result of the second split should be saved in test_Y , val_Z like this:

        test_Y, val_Z = train_test_split(test_Y, train_size=0.50)  # 30%/2 = 0.15 e 0.15

this means that the test_Y array will be split into two half and be saved in two new variables named test_Y, val_Z.

minooei · 2021-01-19T16:14:16Z

@minooei Thanks. I ran and installed it on the terminal: pip install funcy
However when I ran again hoping to generate the three .json, respectively for train, test and val.
(1) This error appeared:
(base) C:\Users\ribei\OneDrive\Documentos\fish_segmentation_detectron2\fish_segmentation_instance\fish_segmentation_instance\Dataset-COCO-Split>python coco_split_train_test_val.py -s 0.6 coco_annotations.json train.json test.json val.json
Traceback (most recent call last):
  File "coco_split_train_test_val.py", line 59, in <module>
    main(args)
  File "coco_split_train_test_val.py", line 32, in main
    info = coco['info']
KeyError: 'info'

you have had this problem one month ago and I answered back then.

michaelmohamed · 2022-02-10T02:18:41Z

Please see my https://github.com/e1-io/echo1-coco-split library, based on this library. I've added support for train, valid, test, and a seed parameter for generation. @minooei , I've also fixed the key errors. This is published to pypi.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

unrecognized arguments and info = coco['info'] #1

unrecognized arguments and info = coco['info'] #1

favervn commented May 15, 2020

nofficer commented May 21, 2020

morganaribeiro commented Dec 9, 2020

minooei commented Dec 14, 2020

morganaribeiro commented Dec 14, 2020

morganaribeiro commented Dec 14, 2020

minooei commented Dec 14, 2020

morganaribeiro commented Dec 14, 2020

minooei commented Dec 14, 2020

morganaribeiro commented Dec 14, 2020

minooei commented Dec 14, 2020

morganaribeiro commented Jan 18, 2021

minooei commented Jan 18, 2021

morganaribeiro commented Jan 18, 2021 via email

minooei commented Jan 18, 2021

morganaribeiro commented Jan 18, 2021 via email

morganaribeiro commented Jan 19, 2021

minooei commented Jan 19, 2021

minooei commented Jan 19, 2021

michaelmohamed commented Feb 10, 2022

unrecognized arguments and info = coco['info'] #1

unrecognized arguments and info = coco['info'] #1

Comments

favervn commented May 15, 2020

nofficer commented May 21, 2020

morganaribeiro commented Dec 9, 2020

minooei commented Dec 14, 2020

morganaribeiro commented Dec 14, 2020

morganaribeiro commented Dec 14, 2020

minooei commented Dec 14, 2020

morganaribeiro commented Dec 14, 2020

minooei commented Dec 14, 2020

morganaribeiro commented Dec 14, 2020

minooei commented Dec 14, 2020

morganaribeiro commented Jan 18, 2021

minooei commented Jan 18, 2021

morganaribeiro commented Jan 18, 2021 via email

minooei commented Jan 18, 2021

morganaribeiro commented Jan 18, 2021 via email

morganaribeiro commented Jan 19, 2021

minooei commented Jan 19, 2021

minooei commented Jan 19, 2021

michaelmohamed commented Feb 10, 2022