当前位置：首页 > news >正文

Python机器学习中的字典列表特征提取

news 2025/5/9 14:50:28

引言

在机器学习项目中，数据预处理是至关重要的一环。当我们的原始数据以字典或字典列表的形式存在时，如何有效地从中提取特征就成为了一个关键问题。本文将从机器学习应用的角度，详细介绍如何使用Python对字典列表进行特征提取，适合初学者系统学习。

一、为什么需要特征提取？

在机器学习中，特征是指用于描述数据的属性或变量。好的特征能够：

更好地表示数据本质
提高模型性能
减少计算复杂度

字典列表是Python中常见的数据结构，特别是在处理JSON格式数据或从NoSQL数据库获取数据时。例如：

users = [{"name": "张三", "age": 25, "gender": "男", "purchases": [12, 45, 32]},{"name": "李四", "age": 30, "gender": "女", "purchases": [56, 23]},{"name": "王五", "age": 28, "gender": "男", "purchases": [78, 91, 12, 45]}
]

二、基础特征提取方法

1. 提取标量特征

# 提取年龄列表
ages = [user["age"] for user in users]
print(ages)  # 输出: [25, 30, 28]# 提取性别列表（可用于标签编码或独热编码）
genders = [user["gender"] for user in users]
print(genders)  # 输出: ['男', '女', '男']

2. 提取统计特征

# 计算每个用户的平均购买金额
avg_purchases = [sum(user["purchases"])/len(user["purchases"]) for user in users]
print(avg_purchases)  # 输出: [29.666..., 39.5, 56.5]# 计算最大购买金额
max_purchases = [max(user["purchases"]) for user in users]
print(max_purchases)  # 输出: [45, 56, 91]

三、高级特征工程技术

1. 处理分类特征

from sklearn.preprocessing import LabelEncoder# 将性别转换为数值
encoder = LabelEncoder()
gender_numeric = encoder.fit_transform(genders)
print(gender_numeric)  # 输出: [1 0 1] (取决于编码器)

2. 创建交互特征

# 创建年龄和平均购买的交互特征
interaction_feature = [age * avg for age, avg in zip(ages, avg_purchases)]
print(interaction_feature)  # 输出: [741.666..., 1185.0, 1582.0]

3. 时间序列特征提取

假设我们有时间序列数据：

transactions = [{"user": "A", "purchases": [{"amount": 100, "timestamp": "2023-01-01"}, {"amount": 200, "timestamp": "2023-01-05"}]},{"user": "B", "purchases": [{"amount": 50, "timestamp": "2023-01-02"}]}
]# 提取最近购买天数（假设今天是2023-01-10）
from datetime import datetimedef days_since_last(purchases):last_date = max([datetime.strptime(p["timestamp"], "%Y-%m-%d") for p in purchases])return (datetime.now() - last_date).dayslast_purchase_days = [days_since_last(user["purchases"]) for user in transactions]
print(last_purchase_days)  # 输出: [5, 8]

四、使用pandas进行结构化特征提取

pandas是机器学习中不可或缺的工具：

import pandas as pd# 转换为DataFrame
df = pd.DataFrame(users)# 添加新特征
df["purchase_count"] = df["purchases"].apply(len)
df["total_spent"] = df["purchases"].apply(sum)
df["avg_spent"] = df["total_spent"] / df["purchase_count"]print(df[["name", "purchase_count", "total_spent", "avg_spent"]])

输出结果：

  name  purchase_count  total_spent  avg_spent
0   张三              3           89  29.666667
1   李四              2           79  39.500000
2   王五              4          226  56.500000

五、文本特征提取

当字典中包含文本数据时：

reviews = [{"product": "手机", "text": "非常好用，电池续航强", "rating": 5},{"product": "耳机", "text": "音质一般，但价格便宜", "rating": 3},{"product": "电脑", "text": "运行速度快，散热好", "rating": 4}
]# 使用TF-IDF提取文本特征
from sklearn.feature_extraction.text import TfidfVectorizertexts = [review["text"] for review in reviews]
vectorizer = TfidfVectorizer()
text_features = vectorizer.fit_transform(texts)
print(vectorizer.get_feature_names_out())  # 输出特征词列表
print(text_features.toarray())  # 输出TF-IDF矩阵

六、处理嵌套字典结构

复杂数据结构特征提取示例：

employees = [{"id": 1,"name": "Alice","skills": ["Python", "SQL", "ML"],"projects": [{"name": "A", "duration": 6, "role": "lead"},{"name": "B", "duration": 3, "role": "member"}]},{"id": 2,"name": "Bob","skills": ["Java", "C++"],"projects": [{"name": "B", "duration": 4, "role": "member"}]}
]# 提取特征：技能数量、项目数量、总项目时长、是否担任过lead
features = []
for emp in employees:feature = {"skill_count": len(emp["skills"]),"project_count": len(emp["projects"]),"total_duration": sum(p["duration"] for p in emp["projects"]),"has_lead_role": any(p["role"] == "lead" for p in emp["projects"])}features.append(feature)feature_df = pd.DataFrame(features)
print(feature_df)

七、特征选择与评估

提取特征后，我们需要评估特征的重要性：

from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import make_classification# 假设我们有一个分类任务
X = feature_df
y = [1, 0]  # 假设的标签# 使用随机森林评估特征重要性
model = RandomForestClassifier()
model.fit(X, y)
importance = model.feature_importances_for col, imp in zip(X.columns, importance):print(f"{col}: {imp:.2f}")

八、实战案例：客户流失预测

假设我们要预测客户流失，原始数据如下：

customers = [{"id": 101,"demographics": {"age": 35, "gender": "F", "income": 50000},"activity": {"logins": 12, "clicks": 45, "purchases": 3},"service": {"plan": "premium", "tenure": 24, "support_calls": 2},"churned": False},# 更多客户数据...
]

特征提取策略：

def extract_features(customer):return {"age": customer["demographics"]["age"],"is_female": customer["demographics"]["gender"] == "F","income_k": customer["demographics"]["income"] / 1000,"login_freq": customer["activity"]["logins"] / 30,  # 假设是30天的数据"purchase_rate": customer["activity"]["purchases"] / customer["activity"]["clicks"],"is_premium": customer["service"]["plan"] == "premium","tenure": customer["service"]["tenure"],"support_per_month": customer["service"]["support_calls"] / (customer["service"]["tenure"] / 12)}features = [extract_features(c) for c in customers]
X = pd.DataFrame(features)
y = [c["churned"] for c in customers]