CLIENT PROFILE

• Industry: Technology
• Client: Software and Technology Provider in the HR/Payroll Space
• Solution: Data Governance & Data Management Assessment and Implementation

BACKGROUND

Enabling data-driven decision-making is a key component of maximizing success in today’s business world, regardless of industry or organization. To be effective, data needs to be complete, accurate, and reliable – a clearly defined Data Governance strategy will ensure that is the case.

Without an effective Data Governance strategy, there are likely to be insufficient or ineffective data policies and procedures. This can lead to poor data quality and ineffective decision making, as insights into patterns, preferences, issues, root causes, and associations could be incorrect.

To build a reliable platform, companies need to start with a clearly defined Data Governance strategy, which then becomes the primary driver to enable the implementation of effective Data Management across the enterprise.

Once implemented, a Data Governance strategy will ensure accurate and trustworthy data, which will then guarantee the tools to support data virtualization and/or visualization are reliable and impactful. This allows companies to begin to focus on more advanced data strategies around artificial intelligence (AI) and Advanced Analytics, which can include machine learning, predictive analytics, statistical modeling, etc.

THE PROBLEM

The client, a software and support organization for small businesses, is gearing up for an Initial Public Offering (IPO). They faced the challenge of demonstrating robust data governance practices to satisfy regulatory requirements and attract investor confidence as well as supporting their increased investment in analytical capabilities. Their existing data management policies and decision-making lacked the necessary transparency and accountability for the scrutiny of public markets.

THE CHALLENGE

The client was looking to have organized and standardized approaches to Data Management and Data Governance, which was a strategic end goal and a top priority across leadership at the organization. 

However, with inadequate Data Governance strategy in place, the client was struggling to address prevailing pain points that manifested within reporting, business intelligence, and analytics platforms in the form of conflicting versions of the truth and extensive manual efforts. The client was at risk of failing to achieve its strategic vision.

As a result, there was a strong desire to mature the approach to data to effectively solve current system issues and create a foundation for successfully leveraging data through accelerated development of technologies like data warehousing and business intelligence.

THE SOLUTION

Acknowledging the need for a transformative approach, Kenway launched a phased approach centered around performing a Data Governance Maturity assessment to help the client understand where the gaps were, providing a set of recommendations, building the related roadmap defining a path to address the underlying issues, and implementing the recommendations defined in the roadmap. 

Data Governance Maturity Assessment

The client shared a long-term vision which sought to simplify its data experience but was unsure how to go about implementing the changes needed to deliver such an experience. The organization required a current state assessment to surface underlying issues, determine gaps, and understand root causes for the pain points it was experiencing. 

Assessments are key to understanding existing processes and capabilities. Lack of an assessment can lead to unrecognized gaps and/or missed opportunities to improve certain aspects of Data Governance/Management, ultimately limiting the client’s ability to meet future state goals and align with their vision.

Kenway’s assessment approach included reviewing existing data processes and interviewing various key employees across the Finance, Accounting, Customer Success (Operations), and Business Intelligence business units. The interviews focused on the following general areas:

• Data Processes (Inefficiencies)
• Data and Systems (Technology)
• Data Quality and Profiling
• Current State of Data
• Legacy Business Processes
• Importance of Data Governance

The Kenway team then synthesized the findings from the stakeholder interviews and workshop to document the current state high-level, data-related processes and procedures and the related pain points encountered across the organization.

The current state was then assessed based on Kenway’s Data Governance assessment criteria:

Rankings for each of the criteria were aggregated and aligned to the Data Governance Maturity Curve for the organization, which provided the client with insight on their Maturity Level.

With the assessment completed and the Data Governance Maturity defined, Kenway was able to provide the client with a set of recommendations and a related roadmap to help address the pain points and achieve its strategic goals while increasing its Data Governance Maturity. This led into the second stage of the engagement, the implementation.

Recommendations & Road mapping

Based on the findings, Kenway ultimately defined a set of key recommendations for the client:

• Stand Up Data Governance Decision-Making and Execution

• Develop Enterprise Change Management Plan

• Develop Report & Query Governance

• Improve Data Policies

• Create Documentation Artifacts

• Support Data Literacy

A roadmap was then established to define a path toward implementing change that would address the recommendations and deliver meaningful and measured value over time. This is shown below:

Implementation

Taking the roadmap from the assessment stage, the Kenway team began the Data Governance implementation by establishing, assessing, reviewing, and validating the foundation. This started by securing organizational buy-in and authority. Kenway took a bird’s eye view, examining at an enterprise level, to understand who within the organization had its best interests in mind and held adequate coverage across the enterprise to enable decision-making capabilities. Kenway facilitated conversations to ensure groups identified would work well together. Kenway facilitated conversations to ensure groups identified would work well together.

Once a Data Governance Charter was defined, and ideal candidates had been selected to create the Steering Committee, Kenway proceeded to create an Operating Model and establish key success metrics. This choreography is captured in the diagram below.

Identifying and onboarding the right participants across the enterprise with decision-making capabilities to represent the Steering Committee posed a hurdle as decision-makers have limited capacity to provide. Kenway entrusted these individuals by communicating the importance of Data Governance within the organization, having numerous conversations with each in alignment with a strategic change management plan comprised of risk assessments, and necessary training and communication plans to aid key decision makers on the committee.

Once the foundation was in place, the next stage of the initiative was to identify one high priority problem or opportunity for the organization to tackle. The diagram below outlines the circular art of accelerating, refining, and reaching market success by solving business problems at hand.

Throughout the journey, obstacles to ensuring the Steering Committee could function together and make decisions in a timely manner were faced. To mitigate this challenge, Kenway facilitated the conversations to ensure folks were examining all considerations, and everyone was confidently representing their scope of business and ultimately reaching a decision unanimously. Kenway shared industry best practices, and expertise from prior implementations at other clients, providing the Steering Committee with unique perspectives on what went right and what went wrong at these other firms.

Kenway also leveraged an in-house decision-making framework to aid the Steering Committee to make decisions that allowed for open and candid feedback with the structured decision-making process. Kenway adapted its own style to align with the existing framework that was more familiar at the organization.

RESULTS

• Defined and instantiated Data Governance Decision-Making and execution:

The definition of a Data Governance Charter outlining roles and responsibilities, along with the formation of a Steering Committee and the implementation of a Data Governance Operating Model have empowered the client with the capacity to streamline data-based decisions and gain confidence in their data

• Incorporated Change Management Practices:

Implementation and BAU Data Governance Activities are now supported by change management practices including a Training Plan, Communication Plan, and Change Management Strategy

• Improved data policies and documentation artifacts and enhanced data literacy:

The client’s systems and policies are updated and aligned with Data Policies with support of an extensive documentation comprising Data Lineage Diagram, Data Catalog, Data Classification, System Flow Diagram. This has permitted the generation of enterprise-wide data ownership and accountability Path to Growth:

In collaboration with Kenway, the client has a comprehensive backlog of opportunities to tackle in subsequent iterations, as well as Data Governance talent acquisition job descriptions and recommendations, both internally and externally

• Internal promotion: 

The client did an excellent job of marketing their initial successes with an enterprise level communication plan, outlining the progress made, steps to maturity, and a plan for continued growth.  

CONCLUSION

The effective implementation and operationalization of a data driven decision making framework highlights the importance of strategic collaboration, technical expertise, and adaptive Data Governance and Data Management policies.

The results of this synergy have allowed the client to realize ROI on their technology investments and fully utilize the tools being implemented.

Explore how our tailored approach to data governance and management can transform your organization's decision-making capabilities. Contact us today to discover how we can help you achieve data-driven success.​ 

Client Profile

• Industry: Direct-to-Consumer Online Wine Retail
• Client: A leading online wine retailer known for its captivating storytelling in email marketing
• Solution: Seeking to streamline content creation while maintaining brand voice, they partnered with Kenway to explore the potential of generative AI

Our client cultivated a loyal following of 27,000 subscribers through captivating email marketing that goes beyond product descriptions. Each email is an immersive experience, transporting the reader on a journey centered around a specific wine. Evocative storytelling is penned by a dedicated writing staff who meticulously follow a defined style guide. This unique voice, both informative and engaging, is a key driver of sales and a differentiator in the competitive DTC wine market. However, crafting these compelling narratives was a costly and time-consuming process.  Recognizing the potential of AI to streamline content creation, our client approached Kenway with the objective of maintaining their brand's unique voice and storytelling magic while exploring new avenues for efficiency.

The Problem

While captivating narratives were the backbone of our client's email marketing success, crafting them presented a significant challenge. Each email, meticulously crafted by a dedicated writing team, aimed to be more than just a product description; it was an immersive journey centered around a specific wine. This commitment to storytelling yielded impressive results, fostering brand loyalty and driving sales. However, the very process that fueled their success became a hurdle to further growth.

Maintaining brand consistency and crafting these narratives required a dedicated writing team to research each wine, develop the story, and meticulously adhere to a clearly defined style guide. This meticulous approach, while ensuring brand identity, limited the volume of content our client could produce. Scaling their email marketing efforts with the current process proved to be cost-prohibitive and resource-intensive.

• Storytelling: These narratives weave stories around the featured wine, incorporating details about origin, vintner philosophy, and tasting notes – all delivered in a lively and engaging style.
• Consistent Brand Voice: A clearly defined style guide ensured a consistent brand voice across all emails, a key differentiator for subscribers. Specific elements like sentence structure, tone, and figurative language were meticulously followed.
• Costly & Time-Consuming: While the human writing team consistently produced high-quality content, the current approach presented a challenge in terms of cost-efficiency. The meticulous attention to detail required for each email, including research, writing, and style guide adherence, presents a high fixed cost of content creation.
• AI for Efficiency: Recognizing the potential of AI to streamline content creation, our client approached Kenway with the objective of maintaining their brand's unique voice and storytelling while exploring new avenues for efficiency.

While the captivating narratives were effective, crafting them was a resource-intensive process. The dedicated writing staff's attention to detail, while ensuring brand consistency, was expensive and limited the volume of content that could be produced.  Our client saw potential in AI to bridge this gap. They envisioned a system where AI could create content or at a minimum generate the initial drafts, freeing up their writing team to focus on editing, polishing, and exploring new creative avenues. However, their primary concern was maintaining the unique brand voice that fueled their email marketing success.

The Solution

Before addressing any technical aspects, our first step was to convert dreams and ideas into a workable vision that garnered consensus.  Successful AI implementation requires a shared vision, and for our client, this meant fostering collaboration across various stakeholder groups beyond just the technology team.

We began by thoroughly assessing their current email marketing processes. This in-depth analysis allowed us to map out the necessary workflow adaptations and considerations that would be essential for successfully integrating an AI solution into their content creation pipeline.  This collaborative approach ensured everyone involved had a clear understanding of the project's goals and the impact on their roles.

With a well-defined vision in place, we could then turn our attention to exploring the technical solutions that would bring this vision to life.

Kenway explored two approaches to achieve our client's goals. The first approach involved leveraging a pre-trained, publicly available generative AI model.  We hypothesized that by feeding the model with the style guide, descriptions of successful past narratives, and specific wine details (name, vintage, winery, etc.), we could guide the model towards replicating the brand's desired voice and style.

Our team experimented with various prompt engineering techniques:

1. To guide the model towards replicating our client's brand voice, we experimented with incorporating the style guide directly into the prompts we fed it. This essentially provided the model with a roadmap for writing content that adhered to the specific elements outlined in the guide, like sentence structure, tone, and figurative language. 
2. We further evaluated the model's output by assigning scores based on how well it adhered to these elements from the style guide. This scoring system allowed us to gauge the model's progress in capturing the brand's desired voice and style.
3. We also explored providing the model with examples of past successful narratives and prompting it to analyze and describe their writing styles. Then, we incorporated those descriptors into the prompts for new narratives, hoping to generate content that mimicked the tone and style of those successful examples.
4. Finally, we established an iterative feedback loop. We generated content samples using various prompt engineering techniques and presented them to our client's Chief Marketing Officer and content editor. Their feedback was thoroughly documented and incorporated into the prompts, further refining the model's output.

While the pre-trained model generated factually accurate content, it lacked the precise storytelling elements and brand-specific nuances that were crucial for our client's email marketing. The narratives, while grammatically correct and informative, often fell flat, failing to capture the engaging and evocative style of the human-written narratives.

Recognizing this limitation, we shifted our approach to fine-tuning a custom AI model specifically tailored to replicate their unique writing style.

Fortunately, our client maintained a comprehensive archive of past email narratives, along with corresponding performance metrics such as click-through rates and conversion rates.  

• This rich data provided a valuable foundation for training. Each past narrative was paired with a prompt based on the original wine details, essentially transforming the archive into a vast prompt-response dataset.
• This dataset was then used to fine-tune a Mistral 5B model chosen for its cost-effectiveness, performance, and accessibility.
• The fine-tuning process involved training the model to recognize patterns and relationships within the existing narratives, allowing it to generate new narratives that mimicked the style and voice of the human-written originals.
• Ultimately, the fine-tuned model was deployed on secure AWS servers, ready to generate narratives based on user input.

What We Delivered

With the fine-tuned AI model humming in the background, our focus shifted to user experience. We wanted to ensure the writing staff could seamlessly integrate AI-generated content into their existing workflow.

Here's what we delivered:

• User-Friendly Interface: We designed a single-page web application with a minimalist interface. This streamlined approach minimized distractions and learning curves for the writing team.
• Simple Data Input: Users could easily input essential wine details like name, vintage, winery, and style (e.g., Cabernet Sauvignon, Pinot Noir). These details were then automatically integrated into a prompt sent to the model.
• Direct Integration with Email Drafting: Upon receiving the prompt, the model generated its narrative. This content was then displayed directly on the screen, ready for review.  A convenient copy function allowed users to easily transfer the generated text to their email drafts.

While developing the interface, we explored additional functionalities like:

• Style Guide Adherence Scoring: This feature would have rated the generated content based on how well it adhered to the style guide.
• User-Defined Revisions: We explored an additional feature where the web app offered a secondary input window after displaying the initial content. This allowed users to provide the model with additional instructions on how to modify its response, facilitating a more iterative approach to content creation.

However, for this project, the simple approach was chosen. The focus was on evaluating the raw performance of the AI-generated narratives with only the potential for controlled influence of human editing. This clean data allowed for a more objective assessment of the model's effectiveness.

The Result

Our client conducted a two-week experiment to assess the effectiveness of the AI-generated narratives. Their email subscribers were divided into four groups:

1. Control Group: This group received no email during the experiment, acting as a baseline for measuring the overall impact of email marketing on revenue generation.
2. Human-Written Group: This group received emails written by our client's dedicated writing staff, maintaining the existing content creation process.
3. AI-Written Group: This group received emails where the narratives were entirely generated by the fine-tuned AI model.
4. AI/Human Hybrid Group: This group received emails where the AI model generated the initial narrative draft, which was then edited by the writing staff before being sent to subscribers.

The experiment yielded valuable insights:

• As expected, the control group, which received no emails, performed the worst in terms of revenue and profit. This confirms the importance of email marketing in generating revenue for our client.
• The human-written and AI-only groups performed similarly in terms of key metrics, with the human-written group having a slight edge in total revenue. This suggests that the AI model was able to successfully replicate the core elements of our client's brand voice and storytelling approach.
• When factoring in operational costs, the AI-written group emerged slightly ahead. While there were initial costs associated with model development and infrastructure setup, these were significantly lower compared to the ongoing costs of employing a dedicated writing staff.
• An intriguing finding was the underperformance of the AI/Human hybrid group. This suggests that the human editing process may have inadvertently altered the AI-generated content in a way that diminished its effectiveness. This highlights the importance of further research into the optimal integration of human and AI content creation processes.

Conclusion

While further refinement and experimentation are needed, our client is now positioned to leverage AI to streamline their email marketing content creation process. The fine-tuned model can generate the narratives, freeing up their writing staff to focus on higher-level tasks like concept development, strategic content planning, and human-centric editing.

This case study demonstrates the potential of generative AI to optimize content creation in the marketing and advertising space. With AI Powered content creation, businesses can create compelling and brand-consistent content while improving efficiency and reducing costs.  The success of this collaboration between Kenway and our client paves the way for further exploration of how AI can be used to enhance content creation across various industries.

Ready to unlock the full potential of Artificial Intelligence? Connect with our experts today to get started.