Skip to content

Commit

Permalink
improved docs, error handling, item type methods added
Browse files Browse the repository at this point in the history
  • Loading branch information
dariobauer committed Oct 29, 2021
1 parent 9a77014 commit 8c7cf75
Show file tree
Hide file tree
Showing 5 changed files with 321 additions and 156 deletions.
14 changes: 12 additions & 2 deletions AUTHORS.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,13 @@
# Authors
# Author

* Dario Bauer - https://github.com/dariobauer
* [Dario Bauer](https://github.com/dariobauer)

# Significant Contributors

These people provided significant contributions to the package including new features:

* [Matteo Tenca](https://github.com/Shub77) - async file downloads

# Contributors

Refer GitHub's [contributors page for Graph-OneDrive](https://github.com/dariobauer/graph-onedrive/graphs/contibutors).
18 changes: 12 additions & 6 deletions CHANGES.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,15 @@


* Improved file download to asynchronously download using multiple connections
* Added HTTPX and aiofiles packages as dependencies - issue #11 opened to replace Requests
* Added HTTPX and aiofiles packages as dependencies
* Replaced Requests package with HTTPX (Issue #11)
* Added verbose keyword arguments to download and upload functions
* Improved error handling
* item_type, is_file, is_folder methods added
* Fixed a bug in the copy_item method
* Set the copy_item method to confirm the copy by default
* Documentation updates
* Updates to authors.md to include significant contributor (Shub77)

## Released

Expand All @@ -15,17 +21,17 @@
Released 2021-10-21

* Major improvements to the cli, now uses argparse, docs updated
* Removed access token validation (Issue 4)
* Allowed access token response to continue when no refresh token provided (Issue 6)
* Various dictionary value lookups improved to account for missing keys (Issue 7)
* Removed access token validation (Issue #4)
* Allowed access token response to continue when no refresh token provided (Issue #6)
* Various dictionary value lookups improved to account for missing keys (Issue #7)

### Version 0.0.1a9

Released 2021-10-20

* Improved code typing
* Updated code formatting
* Improved validation of authorization codes and access tokens (Issues 3 & 4)
* Improved validation of authorization codes and access tokens (Issues #3 & #4)

### Version 0.0.1a8

Expand All @@ -36,7 +42,7 @@ Released 2021-10-17
* Moved testing and debugging decorators to /tests
* Updated some formatting using pre-commit hooks
* Docs improved including syntax highlighting
* Improved auth function to not require a session state (PR#1)
* Improved auth function to not require a session state (Pull Request #1)

### Version 0.0.1a7

Expand Down
83 changes: 64 additions & 19 deletions docs/DOCUMENTATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,9 +77,7 @@ pip install https://github.com/dariobauer/graph-onedrive/archive/main.zip

#### Dependencies

The package currently requires [Requests](https://pypi.org/project/requests/), [HTTPX](https://pypi.org/project/httpx/), and [aiofiles](https://pypi.org/project/aiofiles/). These will be installed automatically if using pip as described above.

Note: We are in a phase of replacing Requests completely with HTTPX.
The package currently requires [HTTPX](https://pypi.org/project/httpx/), and [aiofiles](https://pypi.org/project/aiofiles/). These will be installed automatically if using pip as described above.

## Command-line interface

Expand Down Expand Up @@ -143,6 +141,17 @@ The package currently has the option to save refresh tokens for a user. While it

It is however possible to use one config file to host multiple configurations within itself by using the dictionary key described in the config file section.

### Throttling limits

TLDR; for file transfer methods keep max_connections ≤ 16

Depending on your internet connection and the sizes of the files that you and transferring may see increased transfer speeds with more connections than the default. Graph-OneDrive currently will create a new connection for every 1MiB if not limited by the max_connections. If you have a really fast connection having many connections may slow the overall performance.

A word of caution on too many connections - the Graph API may throttle requests. This can be a hard throttle where all existing connections are cut and a cool-down period is enforced.
Throttling can be applied at the user level or the whole organization. [Details on throttling](https://docs.microsoft.com/en-us/graph/throttling) are available, however the [exact limits are not provided](https://docs.microsoft.com/en-us/graph/throttling).

We recommended to not exceed 16 connections for performance and to avoid throttling.

## Package use

### Package import
Expand Down Expand Up @@ -375,6 +384,54 @@ Returns:

* items (dict) -- metadata of the requested item

#### item_type

Returns the item type in str format.

```python
item_type = my_instance.item_type(item_id)
```

Positional arguments:

* item_id (str) -- item id of the folder or file

Returns:

* type (str) -- "folder" or "file"

#### is_folder

Checks if an item is a folder.

```python
item_type = my_instance.is_folder(item_id)
```

Positional arguments:

* item_id (str) -- item id of the folder or file

Returns:

* folder (bool) -- True if folder, else false.

#### is_file

Checks if an item is a file.

```python
item_type = my_instance.is_file(item_id)
```

Positional arguments:

* item_id (str) -- item id of the folder or file

Returns:

* file (bool) -- True if file, else false.

#### make_folder

Creates a new folder within the input folder/root of the connected OneDrive.
Expand Down Expand Up @@ -427,7 +484,7 @@ Copies an item (folder/file) within the connected OneDrive server-side.

```python
item_id = my_instance.copy_item(
item_id, new_folder_id, new_name=None, confirm_complete=False
item_id, new_folder_id, new_name=None, confirm_complete=True, verbose=False
)
```

Expand All @@ -439,11 +496,12 @@ Positional arguments:
Keyword arguments:

* new_name (str) -- optional new item name with extension (default = None)
* confirm_complete (bool) -- waits for the copy operation to finish before returning (default = False)
* confirm_complete (bool) -- waits for the copy operation to finish before returning (default = True)
* verbose (bool) -- prints status message during the download process (default = False)

Returns:

* item_id (str) -- item id of the new item
* item_id (str | None) -- item id of the new item (None returned if confirm_complete set to False)

#### rename_item

Expand Down Expand Up @@ -539,19 +597,6 @@ Returns:

Examples are provided to aid in development: <https://github.com/dariobauer/graph-onedrive/blob/main/docs/examples/>

## Gotchas / Warnings

### Throttling limits

DLDR; keep max_connections ≤ 16

Depending on your internet connection and the sizes of the files that you and transferring may see increased transfer speeds with more connections than the default. Graph-OneDrive currently will create a new connection for every 1MiB if not limited by the max_connections. If you have a really fast connection having many connections may slow the overall performance.

A word of caution on too many connections - the Graph API may throttle requests. This can be a hard throttle where all existing connections are cut and a cool-down period is enforced.
Throttling can be applied at the user level or the whole organization. [Details on throttling](https://docs.microsoft.com/en-us/graph/throttling) are available, however the [exact limits are not provided](https://docs.microsoft.com/en-us/graph/throttling).

We recommended to not exceed 16 connections for performance and to avoid throttling.

## Support and issues

* Support info, feature requests: <https://github.com/dariobauer/graph-onedrive/blob/main/CONTRIBUTING.md>
Expand Down
40 changes: 27 additions & 13 deletions src/graph_onedrive/_cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -267,9 +267,8 @@ def instance(
client_id, client_secret, tenant, refresh_token
)

# Present menu to user and trigger commands
try:
help_info = """
# Command menu for the user
help_info = """
Graph-OneDrive cli instance actions:
u / usage : prints the OneDrive usage
od / onedrive : print the OneDrive details
Expand All @@ -283,7 +282,10 @@ def instance(
dl / download : download a file
ul / upload : upload a file
exit / quit : exit the menu\n"""
print(help_info)
print(help_info)

# Ask for input and trigger commands
try:
while True:
command = input("Please enter command: ").strip().lower()

Expand All @@ -299,6 +301,9 @@ def instance(
).strip()
if folder_id == "":
folder_id = None
elif not onedrive.is_folder(str(folder_id)):
print("The item id is not a folder.")
continue
onedrive.list_directory(folder_id, verbose=True)

elif command in ["de", "detail"]:
Expand All @@ -311,6 +316,9 @@ def instance(
).strip()
if parent_folder_id == "":
parent_folder_id = None
elif not onedrive.is_folder(str(parent_folder_id)):
print("The item id is not a folder.")
continue
folder_name = input("Name of the new folder: ").strip()
response = onedrive.make_folder(folder_name, parent_folder_id)
print(response)
Expand All @@ -334,7 +342,13 @@ def instance(
new_file_name = input("New file name (with extension): ").strip()
else:
new_file_name = None
response = onedrive.copy_item(item_id, new_folder_id, new_file_name)
response = onedrive.copy_item(
item_id,
new_folder_id,
new_file_name,
confirm_complete=True,
verbose=True,
)
print(
f"Item was copied to folder {new_folder_id} with new item id {response}"
)
Expand All @@ -352,7 +366,10 @@ def instance(
print(f"Item {item_id} was successfully removed.")

elif command in ["dl", "download"]:
item_id = input("Item id to download: ").strip()
item_id = input("File item id to download: ").strip()
if onedrive.is_folder(item_id):
print("Item id is a folder. Folders cannot be downloaded.")
continue
print("Downloading...")
response = onedrive.download_file(item_id, verbose=True)
print(
Expand All @@ -372,6 +389,9 @@ def instance(
).strip()
if parent_folder_id == "":
parent_folder_id = None
elif not onedrive.is_folder(str(parent_folder_id)):
print("The item id is not a folder.")
continue
response = onedrive.upload_file(
file_path, new_file_name, parent_folder_id, verbose=True
)
Expand Down Expand Up @@ -407,18 +427,12 @@ def instance(
print(onedrive.refresh_token)

elif command in ["exit", "exit()", "quit", "q", "end"]:
if use_config_file:
graph_onedrive.save_to_config_file(
onedrive,
config_path=config_path_verified,
config_key=config_key_verified,
)
break

else:
print("Command not recognized. Use 'help' for info or 'exit' to quit.")

except KeyboardInterrupt:
finally:
if use_config_file:
graph_onedrive.save_to_config_file(
onedrive,
Expand Down
Loading

0 comments on commit 8c7cf75

Please sign in to comment.