llm-feat is a Python library that uses OpenAI LLMs (like GPT-4) to automatically generate feature engineering code for pandas DataFrames. You provide your DataFrame and metadata describing what each column means, and the LLM generates context-aware feature engineering code that actually makes sense for your domain.
The library works directly in Jupyter notebooks - when you call the function, the generated code automatically appears in the next cell. You can also get detailed reports explaining the rationale behind each feature, which helps you understand what the LLM is thinking and why certain features were created.
Under the hood, it uses GPT-4's understanding of domain context to generate features that are specific to your problem. For example, when tested on a medical dataset, it generated clinically relevant features like lipid ratios (LDL/HDL) and BMI interactions that a generic rule-based library wouldn't know to create.
*Target Audience:*
This library is designed for:
- Data Scientists and ML Engineers building predictive models who want to speed up the feature engineering process without sacrificing domain relevance.
- ML Practitioners working on real projects who need production-ready tools (I've been using it in my own work), especially useful during the exploratory phase when you're trying to figure out what features might work.
- Anyone tired of manually engineering features and wants an intelligent assistant that understands context rather than just applying generic transformations.
*Comparison:*
vs. Rule-based libraries (featuretools, tsfresh): These libraries use predefined transformation rules that work across all domains but don't understand context. llm-feat uses LLMs to understand your specific domain and generate features that are relevant to your problem. For example, on a medical dataset, it generated lipid ratios and composite risk scores that a generic library wouldn't create.
vs. AutoML tools (AutoGluon, H2O AutoML): AutoML tools are black boxes that handle the entire ML pipeline. llm-feat gives you the actual code to review, modify, and understand. You maintain full control over your feature engineering process while getting intelligent suggestions.
vs. Manual feature engineering: Obviously much faster - what would take hours of domain research and coding happens in seconds. Plus, the LLM often suggests features you might not have thought of.
*Results:*
Tested on the Diabetes dataset:
- Baseline: RMSE 54.33 with 10 original features
- With LLM features: RMSE 53.53 with 20 features (10 original + 10 generated)
- Improvement: 1.47% RMSE reduction, R² improved from 0.44 to 0.46
The generated features included lipid ratios, BMI interactions, and composite risk scores that were clinically relevant and improved model performance.
srinivaskumarr•1mo ago
llm-feat is a Python library that uses OpenAI LLMs (like GPT-4) to automatically generate feature engineering code for pandas DataFrames. You provide your DataFrame and metadata describing what each column means, and the LLM generates context-aware feature engineering code that actually makes sense for your domain.
The library works directly in Jupyter notebooks - when you call the function, the generated code automatically appears in the next cell. You can also get detailed reports explaining the rationale behind each feature, which helps you understand what the LLM is thinking and why certain features were created.
Under the hood, it uses GPT-4's understanding of domain context to generate features that are specific to your problem. For example, when tested on a medical dataset, it generated clinically relevant features like lipid ratios (LDL/HDL) and BMI interactions that a generic rule-based library wouldn't know to create.
*Target Audience:*
This library is designed for:
- Data Scientists and ML Engineers building predictive models who want to speed up the feature engineering process without sacrificing domain relevance.
- ML Practitioners working on real projects who need production-ready tools (I've been using it in my own work), especially useful during the exploratory phase when you're trying to figure out what features might work.
- Anyone tired of manually engineering features and wants an intelligent assistant that understands context rather than just applying generic transformations.
*Comparison:*
vs. Rule-based libraries (featuretools, tsfresh): These libraries use predefined transformation rules that work across all domains but don't understand context. llm-feat uses LLMs to understand your specific domain and generate features that are relevant to your problem. For example, on a medical dataset, it generated lipid ratios and composite risk scores that a generic library wouldn't create.
vs. AutoML tools (AutoGluon, H2O AutoML): AutoML tools are black boxes that handle the entire ML pipeline. llm-feat gives you the actual code to review, modify, and understand. You maintain full control over your feature engineering process while getting intelligent suggestions.
vs. Manual feature engineering: Obviously much faster - what would take hours of domain research and coding happens in seconds. Plus, the LLM often suggests features you might not have thought of.
*Results:*
Tested on the Diabetes dataset: - Baseline: RMSE 54.33 with 10 original features - With LLM features: RMSE 53.53 with 20 features (10 original + 10 generated) - Improvement: 1.47% RMSE reduction, R² improved from 0.44 to 0.46
The generated features included lipid ratios, BMI interactions, and composite risk scores that were clinically relevant and improved model performance.
*Links & Source:*
GitHub: https://github.com/codeastra2/llm-feat
PyPI: pip install llm-feat
I would love feedback on the API design or suggestions for improvements!