Experimentation is a core part of modern data science. Turning it into a repeatable, business-ready process isn’t so easy. From idea to execution, each step requires judgment, collaboration, and good structure. In this article, we will break down how experimentation is done inside a major global retailer led by insights from Lewis Bruder, Director of Data Science at Foot Locker.
Lewis leads a team that works across marketing, finance, and supply chain to build tools that support both strategy and revenue. His team runs two types of experiments: Proof of Concept (driven by business needs) and R&D (focused on long-term capability). Whether testing product recommendations or exploring language models, they follow a clear path—from initial idea to full production rollout.
The process involves cross-functional input, evolving success metrics, and a strong balance between innovation and maintenance. What follows is a clear, real-world look at how experimentation works inside an enterprise data science team—and how it drives real business value.
Why Experimentation Is Built into the Data Science Workflow
In engineering, you usually know what you’re building. You write the code, test for bugs, and expect a certain result. It’s a straight path: the goal is clear, and the steps to get there are mostly predictable.
Data science doesn’t work that way. It’s built on uncertainty. Even the best ideas need testing before anyone knows if they’ll work. That’s where experimentation comes in. It’s not a side task—it’s the foundation.
Before a data science solution becomes a project, it starts as a test. The team needs to figure out if something is possible, useful, and worth investing in. Whether it’s a new model, tool, or platform, every idea is treated as a working assumption. Experiments help prove—or disprove—that assumption under real constraints like time, cost, and architecture.
This early testing phase is how Lewis Bruder’s team filters good ideas from risky ones. It’s a built-in checkpoint that helps avoid wasted effort later.
Two Experiment Types That Drive the Workflow
Lewis Bruder classifies experiments into two main types: Proof of Concept and Research & Development. Each serves a different purpose but works together to move ideas from testing to impact.
Proof of Concept (PoC)
Some experiments start with a clear business question. These are what Lewis Bruder calls Proof of Concept experiments. They’re built around a simple idea: can this solution drive real value?
A PoC is usually proposed by a business partner or a data scientist working closely with the business team. It tests a specific hypothesis tied to a business outcome—like boosting sales, improving recommendations, or reducing costs. The success of a PoC can often be measured in dollars. For example, if a product recommendation tool increases purchases, the business case is clear.
These experiments are direct, focused, and aimed at proving whether an idea can make a real impact.
Research & Development (R&D)
Not every experiment is about solving today’s problem. Some are about preparing for tomorrow.
R&D experiments aren’t tied to a current business need. Instead, they’re focused on exploring new tools, models, or platforms that might be useful later. Lewis gives an example: his team is exploring large language models without a current use case. The goal is to build understanding and readiness so that when a business need does come up, the team can move faster and smarter.
These R&D efforts also help data scientists grow their skills and stay curious. They may not deliver immediate ROI, but they build long-term capability.
From Idea to Experiment: How Initiatives Take Shape
Ideas for experiments often start with business teams or data scientists working closely with them. These aren’t random suggestions—they usually come from someone who sees a clear opportunity or challenge worth testing. Once an idea takes shape, it’s assigned a product owner. While this role isn’t as demanding as in full-scale projects, it ensures the business stays involved throughout the experiment.
Idea Generation and Ownership
Experiments are team-driven from the start. The product owner works alongside the data science team to help shape goals, evaluate progress, and decide what success looks like. This close partnership keeps experiments focused and relevant to real business needs.
Prioritization Through Strategic Alignment
Not every idea moves forward. Experiments that match the company’s current focus—whether it’s marketing, supply chain, or customer experience—get top priority. Lewis makes those calls by looking at where the business is headed and aligning efforts to support that direction. If time allows, less urgent ideas are explored next.
Measuring Success: From Exploration to Execution
Defining success in experimentation doesn’t happen all at once. It’s a process that evolves as the team learns more about the problem and potential solution.
Early Flexibility in Criteria
At the start, success metrics are often unclear. Lewis explains that during the first one to two weeks, the team focuses on understanding the problem and exploring possible solutions. Only after this discovery phase do they begin shaping what success should actually look like.
That flexibility is intentional. Jumping to fixed metrics too early can create pressure or misalignment. Instead, the team waits to see how the work develops, then defines benchmarks that match the real opportunity.
Defining the Gate to Project Status
Once an experiment starts showing promise, it’s time to apply harder filters. At this stage, the team sets clear thresholds—like whether the solution fits the current architecture, stays within budget, or solves the problem fully.
These checkpoints act as the gate between experimentation and a full-scale project. If the solution meets those conditions, it moves forward. If not, the team scraps it and shifts to the next idea.
Transitioning into Production: From Experiment to Product
Turning an experiment into a real product isn’t just about proving it works. It’s about preparing it for real use—and that means some things will change.
What Changes After the Transition
Success criteria used during experimentation help decide whether something moves forward. But once it becomes a formal project, those original benchmarks may no longer apply. Now it’s about building, scaling, and integrating with real systems.
As Lewis explains, once a project is approved, it gets a budget, a team, and a production timeline. From that point, the focus shifts. The early success measures helped justify the build—but new goals emerge as the product grows.
Handling Real-World Production Variables
Live environments bring new challenges. Sometimes models don’t perform the same way they did during testing. Accuracy can drop, data patterns can shift, and results may need re-tuning. This isn’t failure—it’s part of adapting to production.
In those cases, Lewis’s team adjusts code, updates data sources, or fine-tunes thresholds. These changes are normal. They keep the product useful and aligned with business needs after launch.
Business Relationships Evolve as Experiments Scale
Experiments usually begin with a single business partner. But as they show results, more teams often get involved.
Lewis shares an example: a forecasting system that started with operations. Once it began to show value, the finance team also became interested. This kind of expansion is common. What starts as a small use case can grow into a company-wide solution.
As experiments turn into products, the data science team works with more stakeholders. That visibility creates new opportunities and builds trust across departments. It also helps the team stay aligned with business needs as priorities shift.
Internal R&D Time: Fostering Growth and Autonomy
Lewis believes exploration matters. That’s why his team gets dedicated time each week to experiment beyond day-to-day work.
Scheduled Innovation Time
To support long-term growth, Lewis sets aside regular time each week for his team to explore new ideas. This dedicated R&D time gives data scientists space to dive into tools, platforms, or code without being tied to a business request. There are no strict rules on what they can explore—the goal is to give them freedom.
This open structure keeps the team sharp and ready when new challenges or technologies appear.
Motivation Through Curiosity
Beyond the technical benefits, R&D time helps the team stay motivated. It lets them learn, experiment, and play with ideas that interest them. Lewis sees this as more than just skill-building—it keeps the work fun and meaningful. That energy eventually carries over into business projects, making the team more prepared and more engaged.
Balancing Maintenance and Innovation in Production
As more experiments move into production, the need for maintenance grows—but innovation still has to move forward.
The Current Workload Split
Lewis’s team currently spends about 70% of their time on new projects and 30% on maintenance. It’s a balance that keeps the team focused while still supporting live systems. If something breaks in production, fixing it becomes the top priority. The business depends on those outputs to make decisions, so accuracy and speed matter.
Matching Roles to Personality Types
Not everyone wants the same kind of work. Some team members prefer the stability of support roles. Others enjoy the fast pace and unpredictability of new projects. Lewis sees value in both. It’s about matching people to work that suits them—and building a team that can handle both growth and reliability.
Conclusion
Experimentation isn’t just a phase in data science—it’s a way of working that shapes how teams build, learn, and deliver. What stands out from Lewis Bruder’s approach is how experimentation is treated with purpose at every level. It’s not about chasing trends or testing for the sake of it. It’s about staying aligned with the business, thinking ahead, and building systems that are flexible enough to learn—yet strong enough to scale.
By embedding curiosity into structure and treating exploration as part of the job, Lewis and his team show how experimentation becomes more than a process—it becomes a habit that drives real impact. That mindset is what turns ideas into results.
