We investigate the effect of social media advertising content on customer engagement using a large-scale field study on Facebook. We content-code more than 100,000 unique messages across 800 companies using a combination of Amazon Mechanical Turk and state-of-the-art Natural Language Processing algorithms. We use this large-scale dataset of content attributes to test the effect of social media marketing content on subsequent user engagement defined as Likes, comments, shares, and click-throughs with the messages. We develop methods to account for potential selection biases that arise from Facebook’s filtering algorithm, EdgeRank, that assigns messages non-randomly to users. We find that inclusion of widely used content related to brand-personality like humor, emotion and discussion of the brand’s philanthropic positioning increases consumer engagement with a message. We find that directly informative content like mentions of prices, availability, and product features reduce engagement when included in messages in isolation, but increase engagement when provided in combination with brand-personality related attributes. We also find certain directly informative content such as the mention of deals and promotions drive consumers’ path-to-conversion (click-throughs). Our results suggest therefore that there may be substantial gains from content engineering by combining informative characteristics associated with immediate leads (via improved click-throughs) with brand personality related content that help maintain future reach and branding on the social media site (via improved engagement). Our results inform content design strategies in social media, and the methodology we develop to content-code large-scale textual data provides a framework for future studies on unstructured natural language data such as advertising content or product reviews.