Welcome!

By registering with us, you'll be able to discuss, share and private message with other members of our community.

SignUp Now!

Groupby number of hours in a month in pandas

New member
Joined
Feb 1, 2023
Messages
25
Could someone please guide how to groupby no. of hours from hourly based index to find how many hours of null values are there in a specific month? Therefore, I am thinking of having a dataframe with monthly based index.
Below given is the dataframe which has timestamp as index and another column with has occassionally null values.
timestamprel_humidity
1999-09-27 05:00:0082.875
1999-09-27 06:00:0083.5
1999-09-27 07:00:0083.0
1999-09-27 08:00:0080.6
1999-09-27 09:00:00nan
1999-09-27 10:00:00nan
1999-09-27 11:00:00nan
1999-09-27 12:00:00nan

I tried this but the resulting dataframe is not what I expected.
gap_in_month = OG_1998_2022_gaps.groupby(OG_1998_2022_gaps.index.month, OG_1998_2022_gaps.index.year).count()
I always struggle with groupby in function. Therefore, highly appreciate any help. Thanks in advance!
 
New member
Joined
Feb 2, 2023
Messages
4
If need 0 if no missing value per month create mask by Series.isna, convert DatetimeIndex to month periods by DatetimeIndex.to_period and aggregate sum - Trues of mask are processing like 1 or alternative with Grouper:

Code:
gap_in_month = (OG_1998_2022_gaps['rel_humidity'].isna()
                 .groupby(OG_1998_2022_gaps.index.to_period('m')).sum())

gap_in_month = (OG_1998_2022_gaps['rel_humidity'].isna()
                 .groupby(pd.Grouper(freq='m')).sum())
If need only matched rows solution is similar, but first filter by boolean indexing and then aggregate counts by GroupBy.size:

Code:
gap_in_month = (OG_1998_2022_gaps[OG_1998_2022_gaps['rel_humidity'].isna()]
                 .groupby(OG_1998_2022_gaps.index.to_period('m')).size())

gap_in_month = (OG_1998_2022_gaps[OG_1998_2022_gaps['rel_humidity'].isna()]
 
Top