Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: CrashLoopBackOff DataNode During BulkImport #38895

Open
1 task done
akmalmasud96 opened this issue Dec 31, 2024 · 3 comments
Open
1 task done

[Bug]: CrashLoopBackOff DataNode During BulkImport #38895

akmalmasud96 opened this issue Dec 31, 2024 · 3 comments
Assignees
Labels
kind/bug Issues or changes related a bug triage/accepted Indicates an issue or PR is ready to be actively worked on.

Comments

@akmalmasud96
Copy link

Is there an existing issue for this?

  • I have searched the existing issues

Environment

- Milvus version:v2.5.1
- Deployment mode(standalone or cluster): Aws EKS Cluster
- MQ type(rocksmq, pulsar or kafka):    pulsar
- SDK version(e.g. pymilvus v2.0.0rc2): 2.5.2

Current Behavior

I have been importing the data into the milvus with parquet file, using bulk-import. However while importing, the data node is getting into the CrashLoopBackOff, and from the get_import_progress is showing the status for the files to be "Importing" from the last 5 hours.
also Is it not that if the task is not imported the duration after 15 minutes import task will be expired ( default )?

Expected Behavior

No response

Steps To Reproduce

resp = bulk_import(url="http://@localhost:19530",
                   collection_name = "collectionName",
                   partition_name = "partitionName",
                   files= [[path.parquet]] )


### Milvus Log

Attached are the logs of DataNode, which is in the CrashLoop.
[output.log](https://github.com/user-attachments/files/18281680/output.log)


### Anything else?

_No response_
@akmalmasud96 akmalmasud96 added kind/bug Issues or changes related a bug needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels Dec 31, 2024
@bigsheeper
Copy link
Contributor

/assign

@bigsheeper
Copy link
Contributor

image

@akmalmasud96
Copy link
Author

The format of parquet file which i am importing is as follows,
key: str, embeddings: float16

and when we create the parquet using pymilvus LocalBulkWriter, the type of embeddings is uint8 for float16 vector, does milvus also expect uint8 type for parquet ?

@yanliang567 yanliang567 added triage/accepted Indicates an issue or PR is ready to be actively worked on. and removed needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels Jan 3, 2025
@yanliang567 yanliang567 removed their assignment Jan 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Issues or changes related a bug triage/accepted Indicates an issue or PR is ready to be actively worked on.
Projects
None yet
Development

No branches or pull requests

3 participants