통계 기초 정리 2장: 데이터와 표본분포

If Chapter 1 covered data types and descriptive statistics, Chapter 2 tackles a more fundamental question: "How well does the data we have (the sample) represent the whole (the population)?" This question is the starting point for statistical inference.

This post is based on Chapter 2 of Practical Statistics for Data Scientists.

2.1 Random Sampling and Sample Bias#

Term	Definition
Sample	A subset selected from a larger dataset
Population	The entire dataset or the full set of subjects being analyzed
N / n	Size of the population (N) or the sample (n)
Sample bias	When the selected sample does not accurately represent the population

Sampling Method Comparison#

Method	Description	Characteristics
Random sampling	Fully random selection	Simplest approach; suitable for small-scale studies
Stratified sampling	Divide the population into strata, then sample randomly from each stratum	Guarantees representation of minority groups
Simple random sample	Random selection without stratification	A special case of stratified sampling
Sampling with replacement	Selected items are returned to the population before the next draw	Used in bootstrap methods
Sampling without replacement	Once selected, an item cannot be chosen again	Used in typical surveys

import numpy as np
import pandas as pd

np.random.seed(42)

# Population: customer data for 1000 people
population = pd.DataFrame({
    "나이":  np.random.randint(20, 70, 1000),
    "성별":  np.random.choice(["남", "여"], 1000, p=[0.5, 0.5]),
    "구매금액": np.random.lognormal(10, 1, 1000),
})

# Simple random sample (without replacement)
simple_sample = population.sample(n=100, replace=False, random_state=42)

# Stratified sample — preserving gender ratio
stratified = population.groupby("성별", group_keys=False).apply(
    lambda x: x.sample(frac=0.1, random_state=42)
)

print("모집단 성별 비율:\n", population["성별"].value_counts(normalize=True).round(3))
print("\n층화 표본 성별 비율:\n", stratified["성별"].value_counts(normalize=True).round(3))

print(f"\n모집단 평균 구매금액: {population['구매금액'].mean():.0f}원")
print(f"단순표본 평균 구매금액: {simple_sample['구매금액'].mean():.0f}원")
print(f"층화표본 평균 구매금액: {stratified['구매금액'].mean():.0f}원")

In practice: Stratified sampling is essential in user surveys and A/B tests.
When testing a new feature, for example, if you don't match the population's mobile/PC user ratio or the new/returning user ratio, your results will be biased. In medical clinical trials, patients are stratified by age, sex, and severity, then randomly assigned within each stratum.

2.2 Selection Bias#

Term	Definition
Bias	Systematic error — a flaw caused by structural factors, not random chance
Data snooping	Extensively exploring data in search of patterns
Vast search effect	The bias that arises when testing many variable combinations and stumbling upon a result that appears significant purely by chance

The danger of data snooping: Statistically, it is expected that about 5 out of 100 variables will appear significant by chance (at the 5% significance level). Mistaking this for a genuine discovery is a serious error. Any pattern found in training data must be validated on held-out data (a test set).

In practice: Survivorship bias is a classic example.
If you analyze only successful startups to find their "common traits," the companies that shared those same traits but still failed are missing from your data. In recommendation systems, a related problem called exposure bias occurs because only items a user has clicked on are collected as training data.

2.3 Sampling Distributions in Statistics#

Term	Definition
Sample statistic	A value computed from a sample (e.g., sample mean, sample variance)
Data distribution	The frequency distribution of individual observations
Sampling distribution	The distribution of a statistic across many samples
Central Limit Theorem (CLT)	As sample size grows, the distribution of the sample mean converges to a normal distribution
Standard error (SE)	The standard deviation of a sample statistic. $SE = \sigma / \sqrt{n}$

The Central Limit Theorem guarantees that regardless of the shape of the underlying population, if the sample size $n$ is large enough, the distribution of sample means will follow a normal distribution.

$SE = \frac{\sigma}{\sqrt{n}}$

import numpy as np
import matplotlib.pyplot as plt

np.random.seed(0)

# Population: exponential distribution (non-normal)
population = np.random.exponential(scale=5, size=100_000)

sample_sizes = [5, 30, 100]
n_samples = 1000

fig, axes = plt.subplots(1, 3, figsize=(14, 4))

for ax, n in zip(axes, sample_sizes):
    sample_means = [np.mean(np.random.choice(population, n, replace=False))
                    for _ in range(n_samples)]
    ax.hist(sample_means, bins=40, edgecolor="white", color="steelblue", density=True)
    ax.set_title(f"표본크기 n={n}\n표준오차={np.std(sample_means):.3f}")
    ax.set_xlabel("표본 평균")

fig.suptitle("중심극한정리: 표본 크기에 따른 표본 평균의 분포", y=1.02)
plt.tight_layout()
plt.show()

# Theoretical SE vs. simulated SE comparison
print("표본크기  이론 SE      시뮬레이션 SE")
for n in [5, 30, 100, 500]:
    theoretical = population.std() / np.sqrt(n)
    simulated = np.std([np.mean(np.random.choice(population, n)) for _ in range(1000)])
    print(f"n={n:4d}    {theoretical:.4f}      {simulated:.4f}")

In practice: The standard error and the Central Limit Theorem are the foundations for answering "How many users do we need in this experiment?" (sample size determination) in A/B testing.
A larger sample reduces standard error, enabling more precise estimates.

2.4 Bootstrap#

When you have only one sample and cannot repeatedly draw from the population, you treat the sample itself as a stand-in for the population and repeatedly resample from it.

Term	Definition
Bootstrap sample	A new sample drawn from observed data with replacement
Resampling	Repeatedly drawing samples from observed data. Includes bootstrap and permutation (shuffling) methods

import numpy as np

np.random.seed(42)
sample = np.array([2.3, 5.1, 3.8, 7.2, 4.6, 6.1, 3.3, 5.9, 4.2, 6.8])

n_bootstrap = 10_000
bootstrap_means = np.array([
    np.mean(np.random.choice(sample, len(sample), replace=True))
    for _ in range(n_bootstrap)
])

print(f"원래 표본 평균: {sample.mean():.3f}")
print(f"부트스트랩 평균의 평균: {bootstrap_means.mean():.3f}")
print(f"부트스트랩 표준오차: {bootstrap_means.std():.3f}")
print(f"95% 신뢰구간: ({np.percentile(bootstrap_means, 2.5):.3f}, "
      f"{np.percentile(bootstrap_means, 97.5):.3f})")

In practice: Random Forest in machine learning is a bagging (Bootstrap Aggregating) technique that trains a decision tree on each bootstrap sample and then ensembles the results.
When data is scarce and cross-validation becomes unstable, bootstrap confidence intervals can be used to estimate the uncertainty in model performance.

2.5 Confidence Intervals#

Term	Definition
Confidence interval	A range of values expected to contain the true population parameter
Confidence level	The proportion of intervals that would contain the true parameter if the procedure were repeated many times (e.g., 95%)
Interval endpoint	The lower and upper bounds of a confidence interval

A common misconception: "A 95% confidence interval means there is a 95% probability that the parameter lies within this interval" — this is wrong. The correct interpretation is: "If we drew 100 samples using the same method and computed a confidence interval from each, about 95 of those intervals would contain the true parameter."

import numpy as np
from scipy import stats

np.random.seed(42)

# Estimating the population mean confidence interval from a sample
sample = np.random.normal(loc=50, scale=10, size=30)
n = len(sample)
mean = sample.mean()
se = sample.std(ddof=1) / np.sqrt(n)

# 95% confidence interval (using t-distribution — population SD unknown)
ci_95 = stats.t.interval(0.95, df=n-1, loc=mean, scale=se)
ci_99 = stats.t.interval(0.99, df=n-1, loc=mean, scale=se)

print(f"표본 평균: {mean:.2f}")
print(f"표준오차: {se:.2f}")
print(f"95% 신뢰구간: ({ci_95[0]:.2f}, {ci_95[1]:.2f})")
print(f"99% 신뢰구간: ({ci_99[0]:.2f}, {ci_99[1]:.2f})")

# Visualizing confidence level: checking how many of 100 CIs contain the true parameter
true_mean = 50
contains = 0
for _ in range(100):
    s = np.random.normal(true_mean, 10, 30)
    ci = stats.t.interval(0.95, df=29, loc=s.mean(), scale=s.std(ddof=1)/np.sqrt(30))
    if ci[0] <= true_mean <= ci[1]:
        contains += 1

print(f"\n100번 중 모수를 포함한 신뢰구간 수: {contains}개 (기대값: ~95개)")

In practice: Instead of reporting an A/B test result as "conversion rate increased by 2%," you can convey uncertainty by saying "conversion rate increased by 1.2%–2.8% (95% CI)."
If the confidence interval includes 0, the result is not statistically significant.

2.6 Normal Distribution#

Term	Definition
Standardization	Transforming data to have a mean of 0 and a standard deviation of 1
z-score	How far an individual value is from the mean, in standard deviation units
Normal distribution	A symmetric, bell-shaped continuous probability distribution centered on its mean
Standard normal distribution	The normal distribution with $\mu=0$ and $\sigma=1$
QQ plot	A visual check of normality by comparing the quantiles of two distributions

$z = \frac{x - \mu}{\sigma}$

$f(x) = \frac{1}{\sqrt{2\pi\sigma^2}} e^{-\frac{(x-\mu)^2}{2\sigma^2}}$

import numpy as np
import matplotlib.pyplot as plt
from scipy import stats

fig, axes = plt.subplots(1, 3, figsize=(14, 4))

# Normal distribution PDF
x = np.linspace(-4, 4, 300)
for mu, sigma, label in [(0, 1, "μ=0, σ=1"), (0, 2, "μ=0, σ=2"), (2, 1, "μ=2, σ=1")]:
    axes[0].plot(x, stats.norm.pdf(x, mu, sigma), label=label)
axes[0].set_title("정규분포 PDF")
axes[0].legend()
axes[0].set_xlabel("x")

# z-score transformation (standardization)
data = np.random.normal(170, 10, 500)  # Height data (cm)
z_scores = (data - data.mean()) / data.std()
axes[1].hist(z_scores, bins=30, edgecolor="white", color="steelblue", density=True)
x_line = np.linspace(-4, 4, 200)
axes[1].plot(x_line, stats.norm.pdf(x_line), "r-", linewidth=2, label="표준정규분포")
axes[1].set_title("표준화 후 z점수 분포")
axes[1].legend()

# QQ plot — checking for normality
stats.probplot(data, plot=axes[2])
axes[2].set_title("QQ 그림 (정규성 검증)")

plt.tight_layout()
plt.show()

# Applying z-scores: outlier detection
outliers = data[np.abs(z_scores) > 3]
print(f"|z| > 3인 이상값: {len(outliers)}개 ({len(outliers)/len(data)*100:.1f}%)")
# Theoretically, |z| > 3 accounts for about 0.27% of a normal distribution

In practice: In feature scaling, StandardScaler is simply a z-score transformation. Linear regression, SVMs, and neural networks are sensitive to feature scales, so standardization is essential. QQ plots are used to diagnose whether model residuals follow a normal distribution.

2.7 Long-Tailed Distributions#

Term	Definition
Tail	The region of a distribution far from the mean, containing extreme values
Skewness	A measure of the asymmetry of a distribution

$\text{Skewness} = \frac{1}{n}\sum_{i=1}^{n}\left(\frac{x_i - \bar{x}}{s}\right)^3$

Skewness	Meaning
Positive skew (right tail)	Mean > Median; long tail extending to the right
Negative skew (left tail)	Mean < Median; long tail extending to the left
Skewness near 0	Symmetric distribution (like a normal distribution)

import numpy as np
import matplotlib.pyplot as plt
from scipy import stats

fig, axes = plt.subplots(1, 3, figsize=(14, 4))

datasets = {
    "오른쪽 꼬리\n(소득 분포)": np.random.lognormal(0, 1, 2000),
    "대칭\n(정규분포)": np.random.normal(0, 1, 2000),
    "왼쪽 꼬리\n(시험 점수)": -np.random.lognormal(0, 0.5, 2000) + 100,
}

for ax, (title, data) in zip(axes, datasets.items()):
    skew = stats.skew(data)
    ax.hist(data, bins=50, edgecolor="white", color="steelblue", density=True)
    ax.axvline(np.mean(data), color="red", linestyle="--", label=f"평균: {np.mean(data):.1f}")
    ax.axvline(np.median(data), color="green", linestyle="-", label=f"중앙값: {np.median(data):.1f}")
    ax.set_title(f"{title}\n왜도={skew:.2f}")
    ax.legend(fontsize=8)

plt.tight_layout()
plt.show()

In practice: Income, wealth, web traffic, and social media follower counts are all classic long-tail distributions. Using the mean on data like this produces a distorted picture. The standard approach is to apply a log transformation to approximate normality before analyzing, or to use the median as the primary reporting metric.

2.8 Student's t-Distribution#

When the population standard deviation ( $\sigma$ ) is unknown and must be estimated from the sample standard deviation ( $s$ ), additional uncertainty is introduced. The t-distribution is designed to account for this extra uncertainty.

$t = \frac{\bar{x} - \mu}{s / \sqrt{n}}$

Term	Definition
t-distribution	A distribution with heavier tails than the normal distribution; spreads wider for smaller samples
Degrees of freedom (df)	The number of values free to vary; typically $n-1$

As sample size $n$ grows, the t-distribution converges to the standard normal distribution. When $n \geq 30$ , it is generally acceptable to approximate with a normal distribution.

import numpy as np
import matplotlib.pyplot as plt
from scipy import stats

x = np.linspace(-5, 5, 300)

plt.figure(figsize=(8, 4))
plt.plot(x, stats.norm.pdf(x), "k-", linewidth=2, label="정규분포 (n→∞)")
for df, color in [(2, "red"), (5, "orange"), (30, "steelblue")]:
    plt.plot(x, stats.t.pdf(x, df), linestyle="--", color=color, label=f"t분포 (df={df})")
plt.legend()
plt.title("t 분포 vs 정규분포: 자유도에 따른 꼬리 두께")
plt.xlabel("t")
plt.ylabel("밀도")
plt.tight_layout()
plt.show()

# One-sample t-test
np.random.seed(42)
sample = np.random.normal(52, 10, 25)  # True population mean = 52
t_stat, p_value = stats.ttest_1samp(sample, popmean=50)
print(f"t 통계량: {t_stat:.3f}")
print(f"p값: {p_value:.3f}")
print(f"결론: {'귀무가설 기각 (유의미한 차이)' if p_value < 0.05 else '귀무가설 채택 (유의미한 차이 없음)'}")

In practice: The t-test is used in A/B testing to compare the means of two groups. A typical example is testing whether "the purchase conversion rate for the new UI is statistically significantly higher than the original."

2.9 Binomial Distribution#

The distribution of the number of successes when an independent binary (0/1) trial is repeated $n$ times.

Term	Definition
Trial	A single experiment that produces a probabilistic outcome
Bernoulli trial	An independent trial with exactly two possible outcomes: success or failure
Binomial distribution	The probability distribution of the number of successes $k$ out of $n$ trials

$P(X = k) = \binom{n}{k} p^k (1-p)^{n-k}$

import numpy as np
import matplotlib.pyplot as plt
from scipy import stats

fig, axes = plt.subplots(1, 2, figsize=(12, 4))

# Binomial PMF visualization
n, p = 20, 0.3
k = np.arange(0, n+1)
pmf = stats.binom.pmf(k, n, p)
axes[0].bar(k, pmf, color="steelblue", edgecolor="white")
axes[0].axvline(n*p, color="red", linestyle="--", label=f"기댓값 = np = {n*p}")
axes[0].set_title(f"이항분포 (n={n}, p={p})")
axes[0].set_xlabel("성공 횟수 k")
axes[0].legend()

# Real-world example: click-through rate (CTR) test
# 35 clicks out of 1000 ad impressions — is the expected CTR 3%?
observed_clicks = 35
n_impressions = 1000
expected_ctr = 0.03

p_value = stats.binom_test(observed_clicks, n_impressions, expected_ctr, alternative="two-sided")
print(f"관측 클릭 수: {observed_clicks} / {n_impressions}")
print(f"기대 CTR: {expected_ctr*100}%")
print(f"p값: {p_value:.4f}")
print(f"결론: {'기대 CTR과 유의미한 차이 있음' if p_value < 0.05 else '기대 CTR과 차이 없음'}")

# Binomial → normal approximation (when n is large)
n_large = 1000
simulated = np.random.binomial(n_large, 0.3, 10000) / n_large
axes[1].hist(simulated, bins=50, edgecolor="white", color="steelblue", density=True)
axes[1].set_title(f"이항분포 정규 근사 (n={n_large}, p=0.3)")
axes[1].set_xlabel("성공 비율")

plt.tight_layout()
plt.show()

In practice: Estimating ad CTR, defect rate inspection, and email open rate analysis are all binomial distribution problems. When the sample is large enough ( $np \geq 5$ and $n(1-p) \geq 5$ ), you can approximate with a normal distribution to compute confidence intervals.

The distribution of the actual number of events that occur when events happen on average $\lambda$ times per unit of time (or space).

Distribution	Formula	Use Case
Poisson	$P(X=k) = \frac{\lambda^k e^{-\lambda}}{k!}$	Number of events per unit time
Exponential	$f(x) = \lambda e^{-\lambda x}$	Time between events
Weibull	$f(x) = \frac{k}{\lambda}\left(\frac{x}{\lambda}\right)^{k-1} e^{-(x/\lambda)^k}$	Lifespan and failure analysis

$\lambda$ (lambda): The average number of events per unit time. The key parameter in both the Poisson and exponential distributions.

import numpy as np
import matplotlib.pyplot as plt
from scipy import stats

fig, axes = plt.subplots(1, 3, figsize=(14, 4))

# Poisson distribution — number of customer arrivals per hour
for lam, color in [(2, "steelblue"), (5, "orange"), (10, "green")]:
    k = np.arange(0, 20)
    axes[0].plot(k, stats.poisson.pmf(k, lam), "o-", label=f"λ={lam}", color=color)
axes[0].set_title("푸아송 분포")
axes[0].set_xlabel("사건 발생 횟수 k")
axes[0].legend()

# Exponential distribution — time between customer arrivals
x = np.linspace(0, 3, 300)
for lam, color in [(1, "steelblue"), (2, "orange"), (5, "green")]:
    axes[1].plot(x, stats.expon.pdf(x, scale=1/lam), label=f"λ={lam}", color=color)
axes[1].set_title("지수 분포 (사건 간 시간)")
axes[1].set_xlabel("시간")
axes[1].legend()

# Weibull distribution — machine lifespan analysis
x = np.linspace(0, 3, 300)
for k_shape, color, label in [(0.5, "steelblue", "k=0.5 초기고장"),
                               (1.0, "orange",    "k=1.0 지수분포"),
                               (3.0, "green",     "k=3.0 마모고장")]:
    axes[2].plot(x, stats.weibull_min.pdf(x, k_shape), label=label, color=color)
axes[2].set_title("와이블 분포 (수명·고장 분석)")
axes[2].set_xlabel("시간")
axes[2].legend()

plt.tight_layout()
plt.show()

# Real-world example: call center wait time simulation
lam = 3  # Average of 3 inquiries per hour
n_calls = 1000
inter_arrival = np.random.exponential(1/lam, n_calls)  # Time between inquiries (hours)
print(f"평균 문의 간격: {inter_arrival.mean()*60:.1f}분")
print(f"1분 내 다음 문의 올 확률: {(inter_arrival < 1/60).mean()*100:.1f}%")

In practice: In server engineering, requests per second (RPS) follow a Poisson distribution, which forms the basis for planning server capacity using queueing theory. The exponential distribution is used to model equipment MTBF (Mean Time Between Failures), while the Weibull distribution is applied to product warranty period design and predictive maintenance.

The central theme of Chapter 2 is quantifying uncertainty. When inferring about a population from a sample, error is inevitable — and the magnitude of that error is expressed through confidence intervals and standard errors. The p-values in statistical testing, cross-validation in machine learning, and the design of A/B tests all rest on the concepts introduced in this chapter.

// categories

// tags

Statistics Fundamentals Chapter 2: Data and Sampling Distributions