Intercom developed a feature for user engagement via SMS. What metrics would you use to evaluate its success?

This question pertains to product validation and evaluating success through key performance indicators (KPIs). The interviewer is prompting you to reflect on how you would measure the effectiveness and impact of a new feature launched by Intercom—specifically, an SMS engagement tool. You are encouraged to consider the metrics you might employ, the methods for collecting and analyzing the data, and how you would determine whether the feature is successful. Important aspects to consider include the product objectives (such as enhancing engagement, expanding the user base, and increasing satisfaction), the most pertinent metrics for these objectives, strategies for data collection, and approaches for analysis.

To evaluate the effectiveness of Intercom's new SMS feature, I recommend utilizing various KPIs such as the number of users choosing SMS notifications and engagement metrics like response rate, read rate, and click-through rate if links are included in the SMS. Additionally, it would be beneficial to examine customer satisfaction ratings and any alterations in user behavior. For instance, an increase in the usage of other features as a result of SMS notifications would indicate that this feature is positively impacting overall user engagement. Analyzing these metrics before and after the launch of the SMS feature can provide further confirmation of its success.

The effectiveness of Intercom's SMS feature will largely rely on its ability to improve user engagement and experience. To begin with, I would monitor the number of users who sign up for SMS notifications, as this indicates their level of interest. Additionally, I would analyze engagement metrics such as response times, response rates, and whether these interactions result in desired outcomes like sales or bookings. Gathering customer feedback would also be crucial for evaluating the user experience with the feature, which could be achieved through surveys or feedback forms. Ultimately, the success of the SMS feature will be determined by its contribution to Intercom's product objectives.