We introduce an automated, agent-driven approach to the design of photonic devices. We instruct large language models (LLMs) to solve photonic design problems, given access to software tools for performance evaluation (through numerical simulations) and quantitative acceptance criteria (e.g., fabrication rules, geometric constraints, physical-consistency checks). Within this context, agents run autonomous design loops (propose, simulate, evaluate, iterate) and generate devices with state-of-the-art performance. We demonstrate this approach in two stages: First, we run it individually on four canonical problem classes in photonic chip design: a) passive components (waveguide bends, splitters, crossings, etc.); b) active devices (silicon microring modulators (MRMs)); c) radio-frequency (RF) devices (traveling-wave electrodes for a Mach-Zehnder modulator (MZM)); d) chip layout (electrical routing). Then, we combine the previous studies in one demonstration to produce a silicon photonic modulator, incorporating layout, charge transport, optical mode, and RF electrode design. The approach generalizes to any problem that combines a numerical simulator with performance criteria that an LLM can evaluate.