EDA and Feature Engineeringpandas 3.0 · scikit-learn 1.9 · seaborn 0.13 · Python 3.12+
Dashboard
0%
1
Curious builder0 XP earned · 300 to level 2
0 daysFinish a lesson to begin
Badge collection0 of 6 unlocked
72 small wins to finish your pathNext lesson →

Answering a question with groupby

groupby splits a frame into one group per distinct value of the columns you name, and the method you call after it says what to work out inside each group, so one line answers a question that spans two columns.

Last updated: 12 Oct, 2026 · pandas 3.0 · seaborn 0.13 · matplotlib 3.11

Pie and bar charts with a top-N trim counted one column at a time. Most questions name two: a rating band and its colour, a city and whether it has online delivery. That is what groupby is for.

groupby, size and reset_index · from the EDA and Feature Engineering video · 24:21 to 29:10

Syntax: df.groupby(['a', 'b']) returns a GroupBy object, which holds the split but has computed nothing. .size() counts the rows per group, .reset_index() turns the result back into a frame, and .rename(columns={0: 'name'}) names the counted column.

python
grouped = df.groupby(['Rating text', 'Rating color'])   # no work done yet
counts = grouped.size()                 # rows per group, a Series
frame = counts.reset_index()            # back to a frame, count in column 0
frame = frame.rename(columns={0: 'Rating Count'})       # name that column
df.groupby('Rating text')['Aggregate rating'].agg(['min', 'max'])   # other summaries

Loading the Zomato data

Every example here starts from the merged Zomato frame. The zip recipe is the one Reading a CSV that is not UTF-8 built, and the join on the country code is the one Merging a lookup table built. Run this block first, then the example under it.

python
import io, zipfile, urllib.request
import pandas as pd
import matplotlib
matplotlib.use('svg')
import matplotlib.pyplot as plt

url = ("https://raw.githubusercontent.com/krishnaik06/"
       "5-Days-Live-EDA-and-Feature-Engineering/main/Zomatodataset.zip")
zf = zipfile.ZipFile(io.BytesIO(urllib.request.urlopen(url).read()))
with zf.open('zomato.csv') as fh:
    df = pd.read_csv(fh, encoding='latin-1')
with zf.open('Country-Code.xlsx') as fh:
    codes = pd.read_excel(fh)
df = pd.merge(df, codes, on='Country Code', how='left')

Counting rows per group, then getting a frame back

The question: which rating colour goes with which rating band, and how many restaurants are in each pair? Grouping on both columns and counting answers it in one line. size() returns a Series with a two level index, which prints awkwardly and is hard to chart, so reset_index() flattens it into a frame with three columns.

ExampleThe workhorse: groupby, size, reset_index, rename
grouped = df.groupby(['Rating text', 'Rating color'])
print("type before any summary:", type(grouped).__name__)

ratings = grouped.size().reset_index().rename(columns={0: 'Rating Count'})
print(ratings.to_string(index=False))
print("rows in the result:", len(ratings))

What the six groups say

  • Each band has exactly one colour. Six bands, six colours, six rows: the two columns carry the same information twice, which is worth knowing before either is used as a feature.
  • The counts match the single column count. Average 3,737 and Not rated 2,148 are the numbers One chart, one question, one observation charted, which is the check that the grouping did what you meant.
  • The GroupBy object computed nothing on its own. Printing it gives a DataFrameGroupBy, not numbers. The summary method is what runs the work.

Summarising a number inside each group

size() counts rows. Naming a column after the grouping and calling agg works out several summaries of that column per group instead. This is how the rating bands get their actual boundaries, rather than guessing them from the band names.

ExampleThe rating range inside each band
bands = (df.groupby('Rating text')['Aggregate rating']
           .agg(['min', 'max', 'size'])
           .sort_values('min'))
print(bands.to_string())
  • The bands are fixed ranges. Poor is 1.8 to 2.4, Average 2.5 to 3.4, Good 3.5 to 3.9, Very Good 4.0 to 4.4 and Excellent 4.5 to 4.9.
  • Not rated is a single value, 0.0. Its minimum and maximum agree, so those 2,148 rows are not restaurants rated zero; they are restaurants with no rating, stored as a zero.
  • That zero would distort an average. Those 2,148 rows hold a 0.0 that means no rating, so any mean of Aggregate rating taken over the whole column is pulled down by them.

Answering a question about two columns

The question: which countries have restaurants that take online orders? Selecting the two columns, grouping on both and counting gives one row per country and answer pair. Keeping the rows where the answer is Yes is the last step.

ExampleOnline delivery exists in two of the fifteen countries
answer = (df[['Country', 'Has Online delivery']]
          .groupby(['Country', 'Has Online delivery'])
          .size()
          .reset_index()
          .rename(columns={0: 'restaurants'}))
print(answer[answer['Has Online delivery'] == 'Yes'].to_string(index=False))
print("countries in the table:", df['Country'].nunique())

Two countries out of fifteen: India with 2,423 restaurants and UAE with 28. The other thirteen have no row with Yes at all, which is the kind of answer a chart of one column could not have given.

Charting a two column group count

unstack() turns the second level of the group index into columns, which is exactly the shape a grouped bar chart needs: one row per city, one column per answer. fill_value=0 puts a zero where a pair has no rows.

ExampleTwo bars per city, from one groupby
top = df['City'].value_counts().head(5).index
delivery = (df[df['City'].isin(top)]
            .groupby(['City', 'Has Online delivery'])
            .size()
            .unstack(fill_value=0))
print(delivery.to_string())
print()
print("percent of each city's restaurants that deliver:")
print((delivery['Yes'] / delivery.sum(axis=1) * 100).round(1).to_string())

fig, ax = plt.subplots(figsize=(7, 4))
delivery.loc[top].plot.bar(ax=ax, color=['#d64541', '#2e9e5b'])
ax.set_ylabel('restaurants')
ax.set_title('Online delivery in the five busiest cities')
fig.savefig('chart.svg')
A grouped bar chart of the five busiest cities with a red bar for no online delivery and a green bar for yes: New Delhi has 3984 against 1489, Gurgaon 693 against 425, Noida 716 against 364, Faridabad 216 against 35 and Ghaziabad 15 against 10.
  • Gurgaon and Noida deliver most readily, 38.0 and 33.7 percent of their restaurants, against 27.2 percent in New Delhi.
  • Ghaziabad has 25 restaurants in total. Its 40.0 percent is ten restaurants, so the share is not comparable with New Delhi's.
  • Counts and shares answer different questions. New Delhi has by far the most delivering restaurants and the lowest share of them.

size vs count vs value_counts

CallCountsReturns
groupby(c).size()rows per group, missing values includeda Series indexed by the group
groupby(c).count()non-missing values per column, per groupa frame, one column per remaining column
df[c].value_counts()rows per value of one column, NaN droppeda Series sorted by count
groupby(c)[n].agg([...])any summary of column n inside each groupa frame, one column per summary

Where groupby answers the question directly

  • Any question with the word per in it. Sales per region, rating per city, missing values per band.
  • Testing whether missingness depends on something else. Grouping by the target and averaging a missing value flag is the test in Why values go missing.
  • Building an encoding. Ranking categories by the mean of the target inside each group is the whole of Target guided ordinal encoding.
Watch out. groupby drops rows whose grouping value is missing. A count built on a column with nulls therefore adds up to less than len(df) and no warning says so. Pass dropna=False to keep the missing group, and check the total against len(df).
Try it yourself
  • Swap size() for count() in the first example and read why the result has more columns.
  • Group Country and Currency together and find the two countries that share a currency name.
  • Add dropna=False to a groupby on Cuisines and find the group of nine rows it brings back.

This is what real progress feels like.