11 With the first Inside Source response model built, Keri Lee was concerned with how well the model would work in rollout. Would it predict respond- ers, as it suggests?

To answer Keri’s question, her analyst created what is called a gains chart (or lift chart) using the validation sample. She was told by her analyst that such a gains chart will yield a simulation of what she can most likely expect to occur in rollout regarding the model’s predictive strength. Once the pre- dictive strength of the model was confirmed, Keri was ready to select the most profitable names for promotion.

Her goal for the overall campaign was to generate at least 5% profit after overhead. Given such a goal, she determined that she must receive at least a 2.5% response rate. Using the results of the validation gains chart (with predictions), she selected all names meeting her criteria.

It has now been 12 weeks since the promotional mailing occurred. The mailing was slightly under forecast. Keri was anxious to reconcile the fore- casted gains to determine if the model was the cause. Unfortunately, Keri forgot to promote a sample of names not meeting her criteria. As a result, Keri will not be able to determine if the reason the mailing is under fore- cast is due to the regression model not holding up as forecasted or due to the change in promotional format.

As mentioned in Chapter 10, regression models can be built to assist thedirect marketer in predicting which customers on a database are most likely to order, pay, renew, and so on for a particular product offering. Once the final regression model is built for the specified marketing cam- paign, the next step is the actual application of the model to determine

Drozdenko11 2/26/02 6:17 PM Page 217

Drozdenko, Ronald G., and Perry D. Drake. Optimal Database Marketing : Strategy, Development, and Data Mining, SAGE Publications, 2002. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/nyulibrary-ebooks/detail.action?docID=996727. Created from nyulibrary-ebooks on 2020-07-20 13:41:01.

C op

yr ig

ht ©

2 00

2. S

A G

E P

ub lic

at io

ns . A

ll rig

ht s

re se

rv ed

218 OPTIMAL DATABASE MARKETING

which names to promote and which names not to promote from the entire database for an upcoming marketing campaign.

You, as a direct marketer, have several options for selecting names via regression models, some of which are easier to implement than others, depending on the number of models being applied and the flexibility of your computer systems. This book considers two applications:

1. Selecting names based on a single regression “response” gains chart

2. Selecting names based on “expected profit” calculations, which incorporates multiple regression models and marketing cost and profit figures

This chapter reviews the steps required to implement both methods, with detailed examples of the applications in the field of direct marketing, including the development of the marketing forecast.

The Response Gains Chart ________________________________

After building a single regression model to help predict the customers most likely to respond to a particular product offer, you are ready to create an analysis and validation response gains chart to determine (a) model stability and (b) the most profitable customers on the database to promote.

An incremental response gains chart will provide you with a ranking of customers from those most likely to respond to those least likely to respond. On the basis of your marketing cost and profit figures for the product pro- motion under consideration, determine how many names you can promote.

To develop an incremental response gains chart on the analysis portion of the sample, follow these steps:

1. Score the analysis sample using the regression model. This scoring process is done in the same manner as the scoring example in Chapter 10 (see Exhibit 10.5). That is, each name in the analysis sample is passed through the model and a single score assigned to each name, indicating their overall relative likelihood of ordering.

2. Once names are scored, rank the names from highest scoring to lowest scoring.

3. Cut the ranked scores into 10 equal “buckets,” each representing 10% of the sample. For example, Bucket 1 will contain the top scoring 10% of your sample, Bucket 2 will contain the next highest scoring 10% of your sample, and so on. NOTE: In some cases, you may have tied scores at a bucket cutoff level. If this occurs, move your cutpoint for that bucket up (or down) until the score changes. As a result, some buckets may have slightly more (or less) than the desired 10%.

Drozdenko11 2/26/02 6:17 PM Page 218

C op

yr ig

ht ©

2 00

2. S

A G

E P

ub lic

at io

ns . A

ll rig

ht s

re se

rv ed

Gains Charts and Expected Profit Calculations 219

4. For each bucket, calculate the order rate and the gain. The gain is simply the index value minus 100.

To develop an incremental gains chart on the validation portion of the sample, follow these steps:

1. Score the validation (or holdout) sample using the regression model.

2. Once names are scored, rank the names from highest scoring to lowest scoring.

3. Define the 10 buckets as determined on the analysis sample. For example, Bucket 1 on the analysis sample was defined as names having a score greater than or equal to 0.2139. Therefore, the names in Bucket 1 are defined as those names having a score greater than or equal to 0.2139 also (even if that represents more or less than 10% of the validation sample).

4. For each bucket, calculate the order rate and the gain.

ACME Direct test promoted a new product to 20,000 names from the customer segment “5� paid orders in the past 24 months” (see Exhibit 11.1).

The sample was split 10,000 for analysis and 10,000 for validation. At the request of the product manager, the analyst built a regression model (using the analysis sample only) predicting those customers who were most

Entire Customer File Universe Size = 4,670,513

Customers with past purchases that have been

written off Universe Size = 323,156

Remaining customers (the balance)

Universe Size = 3,813,391

Customers who frequently return merchandise (and

have no written-off purchases)

Universe Size = 533,966

Customers with 5+ paid orders in the past 24

months Universe Size = 1,256,479

Remaining Customers (the balance)

Universe Size = 2,556,912

20,000 sample taken from this customer

segment.

Exhibit 11.1 Segment Test Promoted a New Product Offering by ACME Direct

Drozdenko11 2/26/02 6:17 PM Page 219

C op

yr ig

ht ©

2 00

2. S

A G

E P

ub lic

at io

ns . A

ll rig

ht s

re se

rv ed

220 OPTIMAL DATABASE MARKETING

likely to order the product of concern. Once the model was complete, the analyst scored both the analysis and validation samples with the final regression model, ranked the names from highest scoring to lowest scoring, and developed the incremental response gains charts (see Exhibit 11.2) following the steps previously outlined.

Tie scores must have been present, because many of the 10 buckets pro- duced on the analysis sample do not contain exactly 10% of the names.

The gain values displayed in the incremental gains charts are merely a function of the index values. They are calculated as the index to total for each bucket minus 100. Like the index values displayed for univariate tabulations (see Chapter 7), they allow you to more easily view how much better or worse each group or bucket of names performed when compared to the total and in relation to one another.

For the validation sample, the first bucket (those with a score greater than or equal to 0.7546) has a response rate 76% higher than the entire sample. In other words, this regression model has identified roughly 10% of this universe that will respond at a rate 76% higher than the response rate of the entire segment (7.48% vs. 4.25%).

Also notice that both the analysis and validation incremental gains charts have perfect monotonically (perfectly smooth) decreasing response rates when viewed from top to bottom. This is one sign of a good model.

If the incremental gains chart produced on the analysis sample reveals close to monotonically decreasing gains but it does not for the validation sample, it is an indication of a poor model. We advise you to examine the model for weak predictors with high p values or problems with multi- collinearity (see Chapter 10).

Exhibit 11.2 Incremental Response Gains Charts Built on the Analysis and Validation Samples

Incremental Gains on Analysis Incremental Gains on Validation

Gain Gain Score Sample Sample Response Over Sample Sample Response Over

Bucket Level Percent Count Rate Total Percent Count Rate Total

1 GE 0.7546 9.98 998 7.69 81 10.00 1,000 7.48 76 2 0.6532–0.7545 10.02 1,002 6.59 55 10.00 1,000 6.38 50 3 0.5589–0.6531 10.00 1,000 5.61 32 9.94 994 5.53 30 4 0.5013–0.5588 10.05 1,005 5.19 22 10.06 1,006 5.14 21 5 0.4429–0.5012 9.95 995 4.76 12 9.95 995 4.72 11 6 0.4013–0.4428 9.96 996 4.34 2 10.05 1,005 4.42 4 7 0.3521–04012 10.04 1,004 3.02 �29 10.04 1,004 3.19 �25 8 0.2782–0.3520 10.00 1,000 2.30 �46 9.96 996 2.38 �44 9 0.1891–0.2781 10.00 1,000 1.87 �56 9.98 998 2.00 �53 10 LE 0.1890 10.00 1,000 1.15 �73 10.02 1,002 1.28 �70 Total 100.00 10,000 4.25 0 100.00 10,000 4.25 0

Drozdenko11 2/26/02 6:17 PM Page 220

C op

yr ig

ht ©

2 00

2. S

A G

E P

ub lic

at io

ns . A

ll rig

ht s

re se

rv ed

If the incremental gains charts produced on both the analysis and valid- ation samples are not close to monotonically decreasing and are very choppy, it may indicate that the model was not properly coded by your pro- grammer, and therefore, the samples were not properly scored.

As mentioned in Chapter 6, the purpose of a validation sample is to allow the analyst to confirm the analysis results. Remember that a sample is just that, a sample, and as such, a model built on any sample will have error associated with it. How much error your final model may have depends on how careful you were in the exclusion of insignificant or correlated variables. Assessing the model on the validation sample com- pared to the analysis sample allows you to gauge the amount of error associated with your model. If much error is present, we say the model was “overfit.”

How can we tell if the model built reveals an overfitting situation? If the projected gains for the top few buckets are similar between the analysis and validation samples, you have built a stable model. If this is the case, it implies you built little error into the model and it validates nicely on the holdout or validation sample. If the gains are significantly lower for the validation sample when compared to the analysis sample for the top few buckets, then you have a weak model that will not hold up in rollout.

For the gains shown in Exhibit 11.2, the top bucket in the analysis segment responded at a rate 81% higher than the entire sample, whereas the same bucket for the validation sample yielded a response rate 76% higher than the entire sample. This is a 6% ([81�76]/81 � 6%) falloff in gains. Anything greater than a 10% falloff in the top few buckets is cause for concern. A greater than 10% falloff indicates a potential problem with your regression model, and in such cases, we strongly advise you to carefully review your regression model for potential problems (multi- collinearity, weak predictors, etc.).

The only purpose of the incremental gains chart on the analysis sample is to check for falloff in gains when compared to the same on the validation sample. Once you are satisfied that the falloff in gains from analysis to valid- ation is minimal and acceptable, you will base all marketing decisions on the incremental response gains chart produced on the validation sample only. The validation sample provides you with a more stable indicator of what you can expect the model to yield in a rollout situation.

On the basis of the validation gains chart, what percentage of the names can the product manager profitably promote, given the fact that she needs at least a 2.97% response rate to break even? According to the validation gains chart, it appears that by promoting all customers with a regression score above 0.3521 (the top seven buckets), the breakeven criteria will be met. The product manager will not promote the last three buckets of names, because the projected response rates for each are below the required breakeven response rate level.

Gains Charts and Expected Profit Calculations 221

Drozdenko11 2/26/02 6:17 PM Page 221

C op

yr ig

ht ©

2 00

2. S

A G

E P

ub lic

at io

ns . A

ll rig

ht s

re se

rv ed

222 OPTIMAL DATABASE MARKETING

Now that the product manager knows which buckets she will promote, she needs to determine the response rate for these seven buckets combined. To determine this, she will have to create a cumulative gains chart on the validation sample. Cumulative figures tell the associated number of names, the response rate, and the gain for all names falling into a given bucket and all buckets above it. These figures are determined from the incremental gains chart.

It is easy to turn any incremental gains chart into a cumulative gains chart using Excel. The cumulative gains chart shown in Exhibit 11.3 was created in Excel using the incremental gains chart figures created on the validation sample shown in Exhibit 11.2.

For example, the cumulative “Sample Count” for Bucket 2 represents the incremental “Sample Count” for Buckets 1 and 2 combined (1,000 � 1,000). The cumulative “Response Rate” for Bucket 2 represents the incremental “Response Rate” for Buckets 1 and 2 combined: [(1,000 � 0.0748) � (1,000 � 0.0638)]/2,000. The cumulative “Gain” for Bucket 2 is calculated as [(6.93/4.25) � 100] � 100. This information tells the product manager what she can expect if she promotes the top two buckets. Similarly, the cumulative figures for Bucket 3 represent the total names falling into Buckets 1, 2, and 3, the combined response rate for Buckets 1, 2, and 3, and the gains associated with the combined response rate for Buckets 1, 2, and 3. This information tells the product manager what she can expect if she promotes the top three buckets.

The product manager instructs her programmer to score all 1,256,479 customers falling into the “5� paid orders in the past 24 months” segment (see Exhibit 11.1) using the regression model built and to transmit only those

Exhibit 11.3 Cumulative Gains on Validation

Incremental Gains on Validation Cumulative Gains on Validation

Gain Gain Score Sample Sample Response Over Sample Sample Response Over

Bucket Level Percent Count Rate (%) Total Percent Count Rate (%) Total

1 GE 0.7546 10.00 1,000 7.48 76 10.00 1,000 7.48 76 2 0.6532–0.7545 10.00 1,000 6.38 50 20.00 2,000 6.93 63 3 0.5589–0.6531 9.94 994 5.53 30 29.94 2,994 6.46 52 4 0.5013–0.5588 10.06 1,006 5.14 21 40.00 4,000 6.13 44 5 0.4429–0.5012 9.95 995 4.72 11 49.95 4,995 5.85 38 6 0.4013–0.4428 10.05 1,005 4.42 4 60.00 6,000 5.61 32 7 0.3521–04012 10.04 1,004 3.19 –25 70.04 7,004 5.26 24 8 0.2782–0.3520 9.96 996 2.38 –44 80.00 8,000 4.90 15 9 0.1891–0.2781 9.98 998 2.00 –53 89.98 8,998 4.58 8 10 LE 0.1890 10.02 1,002 1.28 –70 100.00 10,000 4.25 0 Total 100.00 10,000 4.25 0 100.00 10,000 4.25 0

Drozdenko11 2/26/02 6:17 PM Page 222

C op

yr ig

ht ©

2 00

2. S

A G

E P

ub lic

at io

ns . A

ll rig

ht s

re se

rv ed

names with a score greater than or equal to 0.3521 to the letter shop for pro- motion. How many names will the programmer be expected to transmit?

If the sample the regression equation was built on is truly a random and representative sample of all names falling into the “5� paid orders in the past 24 months” segment, the programmer should be expected to transmit exactly 70.04% of this universe, or 880,038 names. If the programmer transmits anything significantly higher or lower than this figure, a problem exists such as

♦ The database was not properly scored on the regression model, or ♦ A major shift in the composition of the universe being regressed must

have occurred since the time of the test promotion.

If 880,038 names are transmitted and thus promoted, how many orders should the product manager expect to receive from this campaign? She should expect to receive 46,290 orders (880,038 � 0.0526).

When creating a gains chart, there is nothing magical about 10 buckets. You can have as many or as few buckets as you wish; however, keep in mind that you do not want too few names falling into any one bucket or your estimated response rates for each bucket will have a high error associated with them. In addition, if your final regression model has few variables, you may not have the spread in scores to create 10 buckets and might need to produce your gains chart with fewer buckets. This is typically a problem when regressing “data poor” lists.

Gains Charts and Expected Profit Calculations 223

_________________ Options When Lacking Validation Samples

The importance of a validation sample cannot be stressed enough, espe- cially when an analyst has limited experience in modeling a particular product, list, or offer. If your budget does not allow for test samples large enough to be split into one for analysis and another for validation, two options are available to you.

Historical Gains Falloff Chart

If you have implemented response models in the past and can acquire some historical figures regarding forecasted and actual gains from these past campaigns, you can develop what is called a historical gains falloff chart. These are created by comparing, for several past campaigns, actual to fore- casted gains and determining the falloff observed. Once determined, you can apply this adjustment to future gains charts prior to rollout to ensure that the best possible decision regarding who to promote is made.

Drozdenko11 2/26/02 6:17 PM Page 223

C op

yr ig

ht ©

2 00

2. S

A G

E P

ub lic

at io

ns . A

ll rig

ht s

re se

rv ed

224 OPTIMAL DATABASE MARKETING

For example, last year the product manager at a small newsletter pub- lishing house with two finance titles built a response model to determine which inactive customers she would promote for Hot Stock Tips. Unfortunately, not enough names were tested for the creation of a valid- ation sample. Therefore, the decision to determine who to promote was based on the same sample the model was built on. The product manager, on the basis of an analysis of profit and cost figures, determined she could only promote the top 30% (top three buckets) of the incremental gains chart. For these top three buckets, the cumulative gains chart revealed a gain of 75% in response over promoting everyone. In rollout, she received a gain of only 65%. This was a falloff in gains of 13%. The product manager’s boss was not happy. In all likelihood, if the product manager had had a validation sample available, the gain based on the validation sample would have been closer to what she actually obtained.

Next year, the product manager built a response model to determine which inactive customers she would promote for the other title, The Mutual Fund Report. Again, she lacked a large enough test sample for the creation of a validation sample. Therefore, the decision to determine who to promote was based on the same sample the model was built on. The product manager, on the basis of an analysis of profit and cost figures associated with this title, determined she could only promote the top 30% (top three buckets) of the incremental gains chart. For these top three buckets, the cumulative gains chart revealed a gain of 104% in response over promoting everyone. Is this what she used as her fore- casted gain? Absolutely not. She took advantage of her experience from last year’s campaign and decided to reduce the forecasted gain by 13% (the same loss in gain realized when promoting the same percentage of the file for Hot Stock Tips last year). Instead, the product manager will only forecast a gain of 90.5% (87% of the analysis gain). This is basic- ally how it works. When you lack samples large enough for the creation of a validation sample, you will adjust your gain based on historical information.

Ideally, this product manager will want to determine the historical loss in gains observed for all buckets, not just for Bucket 3 of a cumulative gains chart. And she will do so for several past campaigns. This way, if the prod- uct manager decides to promote, for example, the top six buckets for her next campaign, she will know how much to adjust the gains to ensure a more accurate forecast. In other words, she will want to construct a historical gains falloff chart. Exhibit 11.4 shows forecasted cumulative gains and actual cumulative gains received for two different book titles promoted last year by a small direct marketer of children’s books. In both cases, the forecasted gains were based on the same sample that the model was built on.

Drozdenko11 2/26/02 6:17 PM Page 224

C op

yr ig

ht ©

2 00

2. S

A G

E P

ub lic

at io

ns . A

ll rig

ht s

re se

rv ed

Gains Charts and Expected Profit Calculations 225

Using the calculated falloff in cumulative gains observed from forecast to actual for both products, an average was determined. Using this average falloff in cumulative gains, the product manager is now in a position to know exactly how to adjust each bucket of a cumulative gains chart prior to finalizing future marketing forecasts.

This technique is by no means perfect, but it does give valuable infor- mation to help small direct marketers make better promotional decisions and build more stable forecasts. Falloff in gains is typically a function of an analyst’s experience and that experience in relation to the product, list, and offer being promoted. A more seasoned analyst typically builds models with less falloff than a new analyst.

Bootstrapping

Bootstrapping or bagging is another method that can be employed when samples are too small to be split into one for analysis and another for valid- ation. It is a bit more complex to implement than the gains falloff chart but will yield forecasted gains much closer to actual.

To bootstrap, many subsamples are taken from the main test sample and a regression model built on each. For example, a major entertainment com- pany test promoted a new music club offer to 10,000 names. Rather than build the response model on all 10,000 names, they will instead build many regressions on subsamples of this main sample. They will randomly take 100 subsamples from the main sample of size 1,000 each. For each of these 100 samples, they will build a regression model. Obviously, they will do so using a more automated technique such as the stepwise routine discussed in Chapter 10.

Exhibit 11.4 Assessment of Forecasted Versus Actual Gains

Book Title X Book Title Y Average

Bucket Score (%) Fcst. Actl. Falloff (%) Fcst. Actl. Falloff (%) Falloff (%)

1 10 125 106 –15.00 101 87 –14.00 –14.50 2 20 101 89 –12.00 90 78 –13.00 –12.50 3 30 90 82 –9.00 84 76 –10.00 –9.50 4 40 67 62 –8.00 74 67 –9.00 –8.50 5 50 42 40 –5.00 55 51 –7.00 –6.00 6 60 30 29 –4.00 40 38 –5.00 –4.50 7 70 22 22 –2.00 32 31 –3.00 –2.50 8 80 14 14 –1.00 20 20 –1.10 –1.05 9 90 5 5 0.00 10 10 –1.00 –0.50

10 100 0 0 — 0 0 — —

Drozdenko11 2/26/02 6:17 PM Page 225

C op

yr ig

ht ©

2 00

2. S

A G

E P

ub lic

at io

ns . A

ll rig

ht s

re se

rv ed

Once all models are built, they will examine them and determine which variables consistently entered the models. For the final model, they will choose the variables that consistently entered each model some percentage (e.g., 75%) of the time. By employing this technique, they have avoided the use of any variable that may have been unstable in rollout.

In fact, many direct marketers are closely examining this technique as a replacement for the current analysis and validation methodology. The belief is that the result of a bootstrapped model is a more stable and robust model than a model built on analysis samples and validated on holdout samples.

For more information on bootstrapping, read the paper “On Bagging and Nonlinear Estimation” (May 1999) by Jerome H. Friedman, Stanford University, and Peter Hall, Australian National University (www-stat.stanford.edu/~jhf).

226 OPTIMAL DATABASE MARKETING

Expected Profit Calculations ______________________________

In business decisions, you may not only want to consider the likelihood of an event occurring but also consider the costs and profit figures associated with each of the events. Considering both enables you to base your mar- keting decisions on the likely payoff.

For example, you are promoting a magazine subscription offer for which the cancel rate is high. In determining who to promote, you build both a response model and a payment model to help you select not only customers most likely to order but customers most likely to order and pay. As previ- ously discussed, this is a case in which you will not be able to use a standard response gains chart. This is where the calculation of expected profit comes into play. To determine the expected profit or loss for a particular business scenario, multiply the probabilities associated with each possible outcome by their respective net costs or profit values and then sum.

The expected monetary value calculation (EMV) is written as

EMV � P(O1)M1 � P(O2)M2 � P(O3)M3 � . . . � P(On)Mn

Where P(O1), P(O2), P(O3), . . . P(On) � the probabilities associated with each of

the n possible outcomes of the business scenario and the sum of these probabil- ities must equal 1

M1, M2, M3, . . . Mn � the net monetary values (costs or profit values) associated with each of the n pos- sible outcomes of the business scenario

The easiest way to understand EMV is to review a lottery example. Assume you are considering the purchase of a lottery ticket in which the probability of winning the $1 million jackpot is 1 in 10 million. If

Drozdenko11 2/26/02 6:17 PM Page 226

Homework is Completed By:

Writer	Writer Name	Amount	Client Comments & Rating
ONLINE	Instant Homework Helper 4.8 4305 Orders Completed	$36	She helped me in last minute in a very reasonable price. She is a lifesaver, I got A+ grade in my homework, I will surely hire her again for my next assignments, Thumbs Up! 5.00
Answer.docx Turnitin Report.pdf Contact Writer For Solution Contact Writer For Solution

Order & Get This Solution Within 3 Hours in $25/Page

Custom Original Solution And Get A+ Grades

100% Plagiarism Free
Proper APA/MLA/Harvard Referencing
Delivery in 3 Hours After Placing Order
Free Turnitin Report
Unlimited Revisions
Privacy Guaranteed

Order Now

Order & Get This Solution Within 6 Hours in $20/Page

Custom Original Solution And Get A+ Grades

100% Plagiarism Free
Proper APA/MLA/Harvard Referencing
Delivery in 6 Hours After Placing Order
Free Turnitin Report
Unlimited Revisions
Privacy Guaranteed

Order Now

Order & Get This Solution Within 12 Hours in $15/Page

Custom Original Solution And Get A+ Grades

100% Plagiarism Free
Proper APA/MLA/Harvard Referencing
Delivery in 12 Hours After Placing Order
Free Turnitin Report
Unlimited Revisions
Privacy Guaranteed

Order Now

6 writers have sent their proposals to do this homework:

Writer	Writer Name	Offer	Chat
ONLINE	Financial Solutions Provider I have read your project details and I can provide you QUALITY WORK within your given timeline and budget. 4.7 231 Orders Completed	$32	Chat With Writer
ONLINE	Accounting & Finance Mentor I have done dissertations, thesis, reports related to these topics, and I cover all the CHAPTERS accordingly and provide proper updates on the project. 4.8 1680 Orders Completed	$46	Chat With Writer
ONLINE	Assignments Hut I have read your project description carefully and you will get plagiarism free writing according to your requirements. Thank You 4.9 1428 Orders Completed	$26	Chat With Writer
ONLINE	Assignment Solver I am an experienced researcher here with master education. After reading your posting, I feel, you need an expert research writer to complete your project.Thank You 4.8 1281 Orders Completed	$38	Chat With Writer
ONLINE	Write My Coursework After reading your project details, I feel myself as the best option for you to fulfill this project with 100 percent perfection. 4.7 2205 Orders Completed	$24	Chat With Writer
ONLINE	Ideas & Innovations I am an elite class writer with more than 6 years of experience as an academic writer. I will provide you the 100 percent original and plagiarism-free content. 4.8 1449 Orders Completed	$16	Chat With Writer