Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When doing group_by_dynamic, cannot access "_upper_boundary" in map_groups function. #20731

Open
2 tasks done
lzs-raja opened this issue Jan 15, 2025 · 0 comments
Open
2 tasks done
Labels
bug Something isn't working needs triage Awaiting prioritization by a maintainer python Related to Python Polars

Comments

@lzs-raja
Copy link

lzs-raja commented Jan 15, 2025

Checks

  • I have checked that this issue has not already been reported.
  • I have confirmed this bug exists on the latest version of Polars.

Reproducible example

import datetime
df = pl.DataFrame(
    {
        "x": [
            datetime.datetime(2023, 1, 1, 0, 0, 0),
            datetime.datetime(2023, 1, 1, 0, 1, 0),
            datetime.datetime(2023, 1, 1, 0, 5, 0),
            datetime.datetime(2023, 1, 1, 0, 10, 0),
            datetime.datetime(2023, 1, 1, 0, 11, 0),
        ],
        "y": [1, 2, 3, 4, 5],
    }
).sort("x")
def test_map(group):
    time = group["_upper_boundary"][-1] 
    x = group["x"][-1]
    return pl.DataFrame({"time":[time],"x":x})
agg_df = df.group_by_dynamic(index_column="x", every="5m", period="10m", offset="5m", include_boundaries=True).map_groups(test_map, schema=None)
print(agg_df)

Log output

ComputeError: User provided python function failed: ColumnNotFoundError: "_upper_boundary" not found

Issue description

When doing group_by_dynamic, "agg" can have _upper_boundary if set include_boundaries=True, but "map_group" cannot access this. Though i can't access this, at least add _upper_boundary to the result like what is done in "agg". But the fact is not.

Expected behavior

Either add _upper_boundary to the result like "agg" result, or I can access group["_upper_boundary"][-1] or group._upper_boundary and use them in the function result

Installed versions

--------Version info---------
Polars:              1.19.0
Index type:          UInt32
Platform:            Windows-10-10.0.22621-SP0
Python:              3.9.19 (main, May  6 2024, 20:12:36) [MSC v.1916 64 bit (AMD64)]
LTS CPU:             False

----Optional dependencies----
adbc_driver_manager  
<not installed>
altair               <not installed>
azure.identity       <not installed>
boto3                <not installed>
cloudpickle          <not installed>
connectorx           <not installed>
deltalake            <not installed>
fastexcel            <not installed>
fsspec               <not installed>
gevent               <not installed>
google.auth          <not installed>
great_tables         <not installed>
matplotlib           3.8.4
nest_asyncio         1.6.0
numpy                1.24.3
openpyxl             <not installed>
pandas               2.2.1
pyarrow              <not installed>
pydantic             <not installed>
pyiceberg            <not installed>
sqlalchemy           <not installed>
torch                2.3.0
xlsx2csv             <not installed>
xlsxwriter           <not installed>```

</details>
@lzs-raja lzs-raja added bug Something isn't working needs triage Awaiting prioritization by a maintainer python Related to Python Polars labels Jan 15, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working needs triage Awaiting prioritization by a maintainer python Related to Python Polars
Projects
None yet
Development

No branches or pull requests

1 participant