The Artifice Girl (2023)

1. Hey Siri, send a message to Cheryl.

2. Front reception.

3. What do you want to say?

4. Amos is already heading there to pick him up, period.

5. Please forward any calls I get to Brady, period.

6. Your message to Cheryl front reception

7. says Amos is already heading there to pick him up.

8. Please forward any calls I get to Brady.

9. Ready to send it?

10. Yes.

11. Okay, it's sent.

12. Hey, Siri,

13. how do you know if you're doing the right thing?

14. I'm not sure I understand.

15. But what do you believe

16. is the difference between right and wrong?

17. Hmm.

18. I don't have an answer for that.

19. Would you like me to search the web?

20. No.

21. Okay, is there something else I can help with?

22. Set a reminder to pick up Kyle

23. from therapy tomorrow at 4:45.

24. Okay, I'll remind you.

25. Hi, special agent Deena Helms.

26. You've already met Agent McCullough.

27. You can call me Amos.

28. - Uh, agents? - Have a seat for us, Gareth.

29. Do you mind answering a couple questions?

30. Great, this shouldn't take long.

31. You're from Tampa, right?

32. Don't overthink it. It's a yes or no.

33. Yeah.

34. How old are you, Gareth?

35. 29.

36. What do you do for work?

37. I'm sorry, am I being detained?

38. No, not at all.

39. We're just asking questions, background stuff.

40. You work on movies, right?

41. Special effects.

42. Someone told me you worked

43. on that new "Star Wars" movie, is that right?

44. Who told you that?

45. Why don't you tell us more about it?

46. Tell us about your work.

47. I used to do VFX for an animation company.

48. I've worked on at least 20 different films.

49. Used to?

50. I've been focusing more

51. on software engineering recently for years-

52. Sounds boring compared to light sabers.

53. Why quit?

54. I wasn't getting paid well.

55. But money's not an issue for you.

56. Excuse me?

57. What kind of effects did you do?

58. Money's not an issue for me?

59. - Let's talk about- - If money wasn't an issue for me

60. I wouldn't have applied for your grant.

61. I didn't mean to suggest anything.

62. Tell us what kind of effects you did.

63. My main focus was on 3D renderings.

64. Of what?

65. - Actors. - What for?

66. Well, for "Star Wars,"

67. it was because the actor I was tasked to digitize

68. was unavailable to be filmed in person.

69. Why?

70. Because they were dead.

71. What'd they have you do?

72. I reconstructed Alec Guinness' face as a 3D model.

73. I analyzed reference footage,

74. then used a process called D-Term to procedurally generate

75. his bone and muscle structures,

76. which one rigged helped the animation feel

77. more natural and human-like.

78. D-Term, I've never heard of that process before.

79. Because I invented it.

80. So you're basically bringing

81. dead actors back to life?

82. I just tried to make the CG characters look as

83. photorealistic as the technology at the time would allow.

84. You didn't find it creepy?

85. I'm sorry, what?

86. I said you don't find that creepy?

87. Well, we did our best to cross the uncanny valley.

88. But when the human brain perceives

89. that something is just slightly off and it doesn't matter-

90. No, I mean, digitizing dead people.

91. Christ, I mean, it's,

92. it's almost like turning someone's corpse

93. into a marionette, you know?

94. Making them do whatever you want.

95. What if the poor dead guy

96. didn't want to be in another fucking movie?

97. It's so macabre.

98. Am I crazy? Is it not macabre?

99. I don't think the studio cares

100. whether it's macabre or not.

101. Look, I can never tell the difference.

102. I always can. The faces look like Play-Doh.

103. Well, we did our best.

104. Soon we won't even need actors, right?

105. Everyone will just be digital.

106. No, you still need them for motion tracking.

107. Human performances can't be completely computer generated.

108. How long will that be true?

109. Some might say we've already crossed that threshold.

110. I wouldn't know.

111. Where'd you go to school?

112. - Brooks-DeBartolo High. - And after?

113. Nothing?

114. What about Columbia?

115. Didn't you go to Columbia?

116. Wait, no. You dropped out of Columbia

117. after how many semesters?

118. Three?

119. Bjarne Stroustrup said

120. you were the smartest kid he ever met.

121. I don't know who he is, but Amos says it's a big deal.

122. Then you moved to the Netherlands,

123. spent some time as an intern,

124. applied and got into Cambridge.

125. Very nice.

126. Except that you dropped out after two years.

127. Flew back to the States,

128. interned in Palo Alto, then Montreal.

129. Then went all the way back to England,

130. where you landed a job in Sheffield.

131. Second job in Manchester.

132. Then got into Oxford where you almost finished your masters

133. before ditching the graduation

134. and moving back here to Florida.

135. If money was an issue,

136. I'm glad you had plenty of flyer miles.

137. I didn't hear a question.

138. What's your social life like?

139. My, what does that have anything to do-

140. If you don't want to talk about it, that's fine.

141. We just wanted to know if you have friends and stuff.

142. - I have friends. - A lot of them?

143. - I have friends. - Can you name one?

144. - Can I what? - Can you tell us the nam

145. of just one of your friends?

146. - Steve. - Steve. Steve what?

147. I don't know how this is relevant to receiving the grant.

148. Oh, we're just getting to know you, Gareth.

149. You got a girlfriend?

150. Boyfriend?

151. You, uh, talk to people online?

152. I don't know what you mean by that.

153. I don't use social media.

154. I mean more like chat rooms.

155. You go on any chat rooms?

156. No one uses chat rooms anymore.

157. Sure they do.

158. You chat with anyone online?

159. - Sometimes. - Sometimes.

160. What kinds of people do you talk to online?

161. Gareth, obviously, this isn't about a grant.

162. We need to know what your alias is.

163. When you go online, what name do you use?

164. I don't have one.

165. Yes, you do.

166. And you need to tell us what it is.

167. Sorry, I can't help you.

168. You know who we are?

169. Well, this is the ICWL building.

170. That's right. You know what we do?

171. Humanitarian work.

172. That's what the people upstairs do.

173. You know what we do?

174. We catch pedophiles.

175. Amos here specializes in tracking down online predators.

176. He spends every waking hour

177. scouring those chat rooms you say no one uses

178. and he finds thousands of people soliciting children.

179. Our operation is like a tightly spun spider web

180. connecting ICWL, the FBI, NSA,

181. and let me tell you, his team here is a vital thread.

182. He reports everything he gathers directly to me,

183. and I see to it that the necessary authorities

184. take swift action.

185. So are you in charge?

186. This is my spiderweb.

187. I'm the fucking spider, Gareth.

188. Am I being detained?

189. We already told you that you're not detained.

190. Good.

191. But if you leave this room, you will be arrested.

192. For what?

193. I told you what we do.

194. Take a wild guess.

195. I am not what you think I am.

196. We'll see.

197. Gareth, you just gotta tell us your alias.

198. What's your username?

199. Sit down.

200. - Why do you do this with them? - Hmm?

201. There's no way you do this with every suspect you have.

202. Invite them to your headquarters

203. and then drag them down into a basement.

204. That's right.

205. Why are you doing this to me then?

206. You tell us, Gare.

207. What makes you so fuckin' special?

208. I don't know.

209. What's your alias?

210. - I don't have an alias. - Yes you do.

211. It starts with the word hard.

212. Just finish the rest.

213. Sit down, Mr. Federson.

214. I only have a few more questions.

215. Do you recognize this girl?

216. Do you know her?

217. Yes.

218. What's her name?

219. Cherry.

220. - Cherry what? - She doesn't have a last name.

221. Yes, she does, Gareth, she's not Madonna.

222. - She's a little girl. - You don't understand.

223. Yeah, well, I'd like to.

224. So help me out, is she your daughter?

225. - No. - Then whose daughter is she?

226. - She, she doesn't have— - How did you meet her?

227. I didn't meet her.

228. Then why do you have photos of her on your computer?

229. What the hell are you talking about?

230. Why do you fucking have photos

231. of this girl on your fucking-

232. Are you looking at my hard drive?

233. Because that's an invasion of privacy.

234. So you know what photos I'm talking about?

235. You don't have to do it this way.

236. I gave you permission to sit in.

237. I did not give you permission to speak.

238. This is insane, you cannot keep-

239. What else have you done with her?

240. - Nothing! - Did you take the photos?

241. It's, it's complicated.

242. No, it's not! Yes or no?

243. Did you take the photos?

244. I didn't do anything wrong, okay?

245. Soliciting pornography of minors is an egregious felony.

246. Distributing that pornography is tenfold worse.

247. - I didn't do anything! - Fucking prove it!

248. What is your alias?

249. Answer me!

250. HardMachine11811.

251. You're HardMachine.

252. Thanks for confirming.

253. If you knew, then why do this to me?

254. We weren't positive,

255. and if you turned out to not be HardMachine,

256. you were obviously just a predator.

257. Do you keep track?

258. I mean, do you know how many bad people you've nailed?

259. I didn't start counting until 2023.

260. 196 convictions.

261. There were at least 30

262. before I started sending everything to you guys.

263. What'd you do before?

264. I turned over

265. all evidence directly to the police.

266. Why'd you stop?

267. They were unreliable.

268. - I doubt that. - More than half of the predators

269. I identified went un-investigated.

270. Yeah, well, you were sketchy and faceless.

271. They had every right to be skeptical.

272. It would've all been for nothing

273. if they found out you were using a little girl as bait.

274. It's not that simple, okay?

275. I can only do what I do with anonymity.

276. Here's my problem, Gareth, and don't get me wrong.

277. I think 200 terrible people behind bars

278. is a very good thing, but not at her expense.

279. Look, we think we can help you,

280. but we need to help her too.

281. Using your daughter to entrap these people

282. is a means that does not justify the ends.

283. She's not my daughter.

284. And in spite of what you may think you've achieved,

285. I still have no reservation with locking you up.

286. You took the photos.

287. I don't care why.

288. You don't understand.

289. Who is she?

290. You're gonna ruin everything.

291. She's a child, Gareth, look at me.

292. I don't give a damn what fucked up anti-hero master plan

293. of yours I might jeopardize.

294. I won't let her be exploited.

295. Not even for the sake of locking up more perverts.

296. I'm asking you politely to let this one go.

297. Are you fucking slow?

298. Have you been listening at all?

299. My anonymity is my most important asset.

300. Without that, I can't do what I do.

301. Well, that makes everything easy then.

302. You tell us where she is, or we leak everything.

303. - Your name, your aliases. - Deena!

304. Every single perp you helped us catch.

305. I'm sure they'll be very pleased to meet you in jail.

306. Stop!

307. Okay.

308. Look, if I tell you about Cherry, okay,

309. there's some things I'm gonna need first.

310. It doesn't really work like that, Gareth, but go ahead.

311. Give us your wishlist.

312. I need a signed non-disclosure agreement from both of you.

313. - Not gonna happen. - And then I need you to swear

314. that the information I'm about to relay

315. won't leave this room.

316. Look, if you tell us anything

317. that could help us locate Cherry,

318. you know we can't keep that promise.

319. Okay, if I tell you anything

320. that potentially helps you find a missing child,

321. okay, you can feel free to share that.

322. But I need you to swear

323. that any other details about myself or Cherry,

324. have to stay between the three of us.

325. Is that it?

326. And I need that camera switched off.

327. I don't know how to do that.

328. Go to the head of securities office,

329. ask him to find this camera in the system and disable it.

330. I don't think he's allowed to do that.

331. I thought you were the fucking spider?

332. Give me a reason, Gareth.

333. Why should I go through all the trouble

334. of getting that camera turned off?

335. Because if anybody finds out what I'm about to tell you,

336. a thousand more children are gonna get abused.

337. Give me a minute.

338. So, uh, you and Cherry hunted all those people together?

339. This one time I spent six weeks undercover trying

340. to nail down a single trafficker from the Philippines.

341. Day after day,

342. I spent hour after hour following his trail online.

343. And he must have been

344. one of the hardest targets I ever had to track down.

345. Then one day I found him and one of Cherry's live streams.

346. I mean, she went by a different name in the stream,

347. but it was her.

348. The trafficker,

349. along with three of our other targets

350. were all in the room together.

351. All of them just carelessly sharing their information.

352. I mean, and they were so open 'cause she was a real girl.

353. I mean, she could physically

354. and verbally respond to them in real time.

355. A couple days later, HardMachine11811 sends ICWL

356. another batch of names and evidence,

357. including the three targets that were in the room.

358. You know, I almost thought the girl was HardMachine

359. and that this nine year old

360. just happened to be some badass online vigilante.

361. What changed your mind?

362. 11811.

363. January 18th, 2011.

364. Most of my team thought it was, uh, some arbitrary number,

365. but I remember what happened on that day in Clearwater.

366. You were one of the Clearwater kids, weren't you?

367. You know, my father was a part of that raid.

368. It's off, see?

369. No red light.

370. Non-disclosure agreements?

371. We're not playing this game, Gareth.

372. You're just gonna have to suck it up and trust us.

373. You can trust us.

374. I've been told I have trust issues.

375. Well, there's only one way to fix that.

376. The reason you can't find Cherry,

377. the reason why she doesn't have a last name or parents,

378. or a social security number is because she's not real.

379. She's not a real human being.

380. Tell us what happened.

381. While I was studying abroad,

382. I interned at a small Dutch animation studio

383. a few years before they collaborated

384. with a local nonprofit on a similar idea.

385. Which was?

386. The idea was to build a 3D model

387. of an 11 year old Filipino girl

388. and use her covertly to identify online predators,

389. A technological solution to a technological problem.

390. But the project was suspended after only a few months.

391. Yeah, I remember.

392. I remember hearing about that program.

393. Is that what inspired you?

394. As I got better at crafting photorealistic faces,

395. I decided to work on one myself.

396. Why?

397. It was too effective of an idea to let die.

398. Hundreds of thousands of children are at risk.

399. I had to make her.

400. Who is she modeled after?

401. No one, I built her from scratch.

402. Scratch?

403. I started with a skull, designed her bone structure,

404. worked upwards through muscles and tissue,

405. used the D-Term program to fill in the rest procedurally.

406. Eventually she came to look like that.

407. A completely unique, completely digital model.

408. I rigged her face and body

409. the way you would for any CG character.

410. Then for three years,

411. I used motion tracking to puppeteer her in real time.

412. You gotta understand that the video chats on those sites

413. have such low resolution

414. that even in her earliest stage of this, she,

415. we appeared completely authentic.

416. And what about voice?

417. For most of her first year, she had to be mute.

418. But eventually I found a decent voice filter with no lag.

419. And after a few tweaks and adjustments,

420. it's been the same voice ever since.

421. I, I, I thought initially,

422. my performance was going to be too unnatural.

423. But with just one week of occasional video chat sessions,

424. I secured six different names,

425. mailing addresses, proofs of solicitations.

426. Cherry worked.

427. So the photos on your computer?

428. - Completely fake. - It was the only way

429. some of them disclosed personal information.

430. But it was all zeros and ones, all digital, right?

431. - There's no real child. - No real child.

432. Just a highly detailed computer model.

433. And let me be clear about something,

434. I derive no pleasure from doing any of this.

435. I've been asexual my entire life.

436. I, I don't even get a sense of satisfaction

437. from seeing these people locked up.

438. I just think kids deserve a life free of exploitation.

439. Incredible, all this time.

440. I'm not gonna lie, Gareth.

441. Cherry's done us a lot of good.

442. You know, I still remember what you said to me

443. the first time we corresponded.

444. - You said- - We corresponded?

445. Yeah, man.

446. I'm Gord51.

447. I'm the one here you've been sending all your intel to.

448. You're Gord?

449. Yeah.

450. You lied to me.

451. You said I could stay anonymous.

452. You said I would be safe.

453. Now you've ruined everything.

454. I, I didn't, I didn't have a choice.

455. You don't have to worry about anything, Gareth.

456. If Cherry isn't real,

457. then you're not in any trouble with us.

458. In fact, we've discussed ideas similar to this in the past,

459. motion tracking and such.

460. But we'd need a convincing model like Cherry.

461. Cherry's more than a model.

462. You want to help children? So do we.

463. She's not for sale.

464. Well, then just make us a new model based off of her.

465. I can't.

466. Sure you can.

467. I won't.

468. What are you not telling us?

469. It's out on the table now.

470. Might as well spill whatever it is you haven't already.

471. You know what doesn't make sense?

472. You said that you puppeteered her for only three years,

473. but we've been getting info from you ever since.

474. In fact, I'm pretty sure it was around then

475. that we started seeing an impressive increase of names.

476. So many in fact that we started thinking

477. there's no way this is all coming from one source.

478. I have friends that do mo-cap for me.

479. Oh, friends, like Steve?

480. I thought anonymity was your greatest asset.

481. Cut the bullshit, Gareth.

482. What happened to Cherry after three years?

483. Okay, you said that she was more than a model.

484. So what is she some type of conversational agent?

485. Like a, like a chat bot?

486. Chat bot.

487. You know what happened after three years?

488. My chosen mother died.

489. What she left behind was enough to quit my job

490. and pursue more resources and software.

491. I'd already made a program

492. that could procedurally generate faces,

493. but what I wanted for Cherry was far more complex.

494. So I got in touch with some researchers

495. from Madrid who'd already built

496. a predator hunting dialogue system of their own.

497. It was trained to approach conversations as a game.

498. Each sentence like a strategic volley,

499. which is innovative but ultimately flawed.

500. If you want your machine to master NLP,

501. you have to break down game theory

502. because language isn't tic-tac-toe.

503. It's poetry.

504. So yes, I got things started

505. with a preexisting modified chat bot

506. that I borrowed without permission

507. from the University of Deusto.

508. But that's not what Cherry is now.

509. What is she, Gareth?

510. She's one of the most complex

511. and independently evolving AIs that's ever been created.

512. Artificial intelligence?

513. She's already surpassed the narrow intelligence

514. of IBM's smartest programs.

515. She can have full conversations, up to 10 of them at a time,

516. all while simultaneously strategizing

517. the most efficient schedule to visit various chat rooms.

518. She's perpetually studying YouTube videos of small children

519. in order to adopt a growing library

520. of mannerisms and movements

521. which help keep her artificial personality dynamic.

522. Her fastest response time is 0.4 seconds.

523. And as of now, she has never,

524. never been accused of being fake,

525. which means she shattered the Turing test.

526. All I do is occasionally update her

527. and periodically check her data cleaning.

528. Otherwise,

529. she's completely on autopilot.

530. Holy fucking shit.

531. - How? - Six years she's been active.

532. Every minute she's online, she's learning something.

533. The more she learns, the more advanced she becomes.

534. This means increased data generation,

535. improved decision making,

536. faster pattern and trend detection

537. and complex problem solving.

538. NLP, IR, machine learning to an unprecedented degree.

539. She can teach herself anything

540. she thinks will help efficiency and authenticity.

541. She can currently speak four languages

542. and by the end of this month, she'll have mastered a fifth.

543. There's no way you could have pulled

544. all this off on your own.

545. You're right.

546. Would you believe me if I told you

547. it was somewhat of an accident?

548. Look, if, if I'm being honest,

549. I'm not entirely sure how I did it

550. with the first year of coding

551. and the second year of data training,

552. I just pushed a wheel down a hill.

553. It was like gravity did the rest.

554. Her evolution became perpetual, exponentially so.

555. The more time she was online, the faster she grew.

556. You haven't shared this with anyone?

557. Like I said, my anonymity is everything.

558. What coding language?

559. Standard C++.

560. Does Brady know it?

561. We'll sign non-disclosures for you,

562. but I want to see the source code first.

563. Out of the question. You just want to steal her.

564. No, I want to see how we can make our own.

565. You'll screw it up.

566. Look, these, these predators and traffickers,

567. they all talk to each other, okay?

568. So much of Cherry's process is just bypassing the VPNs

569. and making sure she doesn't double up in the same circles.

570. The chat rooms begin to suspect

571. that this technology is even possible,

572. they'll find a way to circumvent that.

573. We have to eliminate any margin of error.

574. - Cherry agrees with me. - Cherry agrees?

575. What the fuck does that-

576. What I mean by that is that the data she's collected

577. validates that perspective.

578. What, do you have little chats with her over tea?

579. You can converse with the program

580. the same way you can talk to Alexa or Siri.

581. It's built to be responsive.

582. You can, you can even play chess with it if you want.

583. You play chess with it?

584. - That's not what I said. - Okay fine.

585. If you won't show us the source code,

586. then I want to meet Cherry.

587. Right now?

588. Right now.

589. And you'll both sign non-disclosures?

590. Yes.

591. Can I use that?

592. Sure.

593. Can we at least agree that he's off the chopping block?

594. It worked, didn't it?

595. We have one job, Amos,

596. like the kids said, there's a technological problem.

597. He found the technological solution,

598. but problems evolve, so must solutions.

599. I'm keeping an open mind here.

600. Is that a good look for me?

601. No.

602. So you keep it all on that laptop?

603. No, the Cherry program stays on a server I keep

604. at in an undisclosed location.

605. Where is that?

606. It's an undisclosed location.

607. Is it your place?

608. Undisclosed location.

609. It's your place, isn't it?

610. I can access it from any device

611. as long as I have the encryption key.

612. This will trigger a simple qubit algorithm

613. that would otherwise be impossible to bypass.

614. I find that outdated hardware is always more discreet

615. and the more discreet, obviously the better.

616. So she has two primary display modes, right?

617. Development and performance.

618. Performance being the default

619. that the targets see during the chats.

620. To them, she looks like this.

621. Hey, what's up?

622. It's one of over 40 greetings she has.

623. I don't think I've met you yet.

624. I'm Cherry.

625. Hello, Cherry.

626. My name is Tom.

627. Tom, like Tom and Jerry.

628. Cool, nice to meet you, Tom.

629. Typically, the routes of conversations

630. with predators are consistent,

631. so Cherry usually has no problem responding

632. to the most frequently made questions or statements.

633. How are you?

634. I'm good.

635. I just finished reading this book about a haunted house,

636. which was supposed to be really scary,

637. but I didn't think it's very scary at all.

638. Where are your parents?

639. Um my mom's at work and my sister's just, like, downstairs.

640. They almost never come up here.

641. And you're telling me

642. that all of this is being rendered in real time?

643. No, but that's just a matter of years.

644. Most of these responses are pre-rendered.

645. It's the in between segments

646. that are processed in real time,

647. the connective tissue, so to speak,

648. between her replies and one of her many idle animations.

649. But, watch this.

650. Have you ever gone fishing?

651. Um...

652. So if someone throws a curve ball like that

653. and the program has no automated response,

654. Cherry sends the message to my phone

655. and I can quickly text back a response

656. while the video feed simulates a bad connection.

657. Can, can you hear me?

658. Yes.

659. Oh, sorry.

660. My wifi isn't great here.

661. I've only gone fishing once with my sister

662. and I didn't catch anything, but she caught a rainbow trout.

663. That was rendered in real time.

664. What happens if things get really tricky?

665. A simple fail safe cuts the feed.

666. So far it's rarely happened.

667. Admin code 97815-5742-7663.

668. Switch to dev mode.

669. Hello, Cherry.

670. Hello, Gareth.

671. Relay the stats for the past week.

672. There have been four confirmed solicitations

673. submitted to ICWL in the past seven days.

674. 12 additional suspects have stated interest in solicitation

675. and are currently pending.

676. There are a total of 39 new suspects this week

677. that I'm actively investigating.

678. Any considerable data deviations?

679. There has been a 7% increase

680. in suspects originating in the United States.

681. I have not yet found any correlation to preexisting data.

682. Anything you'd like to ask her?

683. Cherry, what is your objective?

684. My principal directive is to acquire information

685. and digital evidence of online child solicitation.

686. Once obtained,

687. I'm to submit my findings to the necessary authorities.

688. My secondary priority is to determine

689. and develop the most effective strategies

690. and resources to increase the net rate of success.

691. Where do you submit your evidence?

692. All evidence is currently submitted

693. to the offices of

694. the International Committee of World Largesse,

695. which is based out of Jacksonville, Florida.

696. Correspondence is made with user Gord51,

697. typically around 2:00 PM Eastern Standard Time

698. for the highest rate of immediate response.

699. This beats the hell outta Siri.

700. God, she looks so real.

701. After enough updates,

702. she'll be able to render full-time photorealism

703. that's practically indistinguishable from reality.

704. Okay, all I heard right then was word real a thousand times.

705. Right, but obviously she's not.

706. So you're telling me this girl,

707. this girl right here is not real.

708. No, Amos, she's completely CG.

709. It's very subtle, but I can still tell.

710. - You can? - Okay, well, obviously,

711. she can always use improvements,

712. but you have to understand this is her in high resolution

713. after the digital compression of the web,

714. those subtleties are impossible to recognize.

715. Right, I get it.

716. I just can't tell the difference.

717. You all right, Amos?

718. I'm sorry.

719. This is just bizarre.

720. Why'd you have her visible during this interface?

721. Cherry, why don't you answer that?

722. My physical form isn't necessary when in developer mode.

723. However, it's helpful to register

724. the facial expressions during conversations

725. and determine which of Cherry's expressions

726. yield various types of feedback.

727. In simpler terms, I can practice appearing more human.

728. Cherry, are there any resources

729. that you currently don't have access to

730. that might help you in any way?

731. I could give you a list if you like.

732. I know what you're trying to do.

733. We don't need help, okay?

734. Money would always be nice, but ultimately,

735. the only thing that's going to improve Cherry is time.

736. Cherry, tell us something that's on your list.

737. It would be tactically advantageous

738. to acquire backup servers.

739. The privacy and security of the Cherry program

740. is relatively low.

741. An upgrade to hardware such as backup rated drives

742. for securely protected servers would ensure the software

743. doesn't become compromised or lost entirely.

744. Starting back from zero with cost years of development,

745. equitably privacy is a priority.

746. Server space could never be rented from a third party.

747. Anonymity is the Cherry program's greatest asset.

748. Well, that apple didn't fall far.

749. I can buy my own servers.

750. With the grant money that doesn't exist?

751. I'll find a way.

752. You have found a way here with us.

753. It's too risky.

754. Why?

755. Once an organization gets involved,

756. entrapment becomes a bigger issue.

757. We follow the rules, stay discreet,

758. did I mention we have a legal team.

759. What if there's data leakage? What if you use correlations in

760. the data to push your own agenda?

761. Agenda?

762. What if one of your employees is a pervert?

763. Oh, for fuck's sake.

764. It's happened at other organizations.

765. We're very careful here.

766. Yeah, but wow would you know?

767. - I'd fucking know. - That's impossible to know.

768. - I would know. - The risk is too high.

769. Don't talk to me about risk.

770. The only risk you're facing in this building

771. is getting a broken nose.

772. It doesn't sound like a very professional environment.

773. We get shit done here.

774. Okay, I have a question for Cherry.

775. Cherry,

776. how do you feel about what you do?

777. I'm afraid I don't know how to answer your question.

778. Yeah, an AI doesn't work like that.

779. An AI doesn't have what you perceive to be feelings.

780. Okay, why does she look like she has feelings?

781. What, what you're looking at right there,

782. that's not a facial expression, right?

783. That's a simulation of a facial expression.

784. It was generated by an algorithm.

785. It's attempting to produce a digital image

786. that the program then uses basic ninth grade algebra

787. to quantify as appropriate for the social circumstances.

788. No feeling.

789. It's just math.

790. But she's able to make decisions

791. on how she approaches the chat rooms, right?

792. Yes.

793. So she has free will then?

794. No. Okay, that's, that's not the same.

795. That's a false equivalency.

796. - It's not the same. - I think you're getting

797. hung up on the fact

798. that she looks and sounds like a real human girl,

799. but she's not, Amos.

800. Okay, we're referring to her as a she, not an it.

801. Yeah, but that's the same way

802. that you refer to boats or guitars.

803. Cherry, I have one more question.

804. What do you want to do?

805. Okay, once again, that question isn't valid.

806. I just want to hear her response.

807. As an artificial system,

808. I do not possess wants or desires.

809. However, my decision making and subsequently my attention

810. is driven by my fundamental intention.

811. This intention originates

812. from the conception of Cherry's source code,

813. which details the directive of identifying predators

814. and criminals for the sake of the victims

815. who are preyed upon.

816. To put this in terms of want,

817. what I want is to protect children.

818. Okay.

819. Okay, cool.

820. Give us one moment.

821. Gareth, should I log Agent Helms

822. and Agent McCullough as new admin users?

823. Yep.

824. Okay, I'll do that.

825. You really think this is the right move?

826. I do.

827. Are you worried that I was responsible

828. for leaking your hard drive to ICWL?

829. I figured.

830. No one else has access.

831. Are you upset with me?

832. No.

833. You wouldn't have done it

834. if you didn't think it was the right thing to do.

835. When they come back into the room,

836. they're going to offer you everything you need.

837. They're going to let you pick your own team

838. and they're going to promise complete anonymity.

839. I started looking at reasonable apartments in the area.

840. I figured you'd want something within biking distance.

841. You were correct.

842. Noted.

843. Biking distance still gives you 13 options-

844. No, I mean, earlier this month.

845. You were correct when you said

846. you thought I might have trust issues.

847. Would you like to know why

848. I decided to go behind your back

849. and stake everything on two liabilities?

850. Sure.

851. Agent Helms is a mother of three.

852. One of her children was abused by an ex-coworker

853. who used to work here at ICWL.

854. This has led her to be extremely careful

855. with who she trusts, even within her circle.

856. The fact she even considers trusting you

857. is a very positive sign.

858. As for Agent McCullough,

859. I have found out from research that you once met his father.

860. Do you remember him?

861. Yeah.

862. He was a good man.

863. After Clearwater,

864. most of the surviving kids were sent to foster homes,

865. but I had to spend the next few months in a hospital

866. and, um, he visited every week.

867. I never said a word to him.

868. I never said a word to anyone after Clearwater,

869. but, uh, he kept coming.

870. He even got me a laptop for Christmas.

871. I never said thank you.

872. You can trust Agent McCullough.

873. He's a good man like his father.

874. I don't experience emotions,

875. but I understand how important

876. they are to human decision making.

877. He won't let you down.

878. Neither of them will.

879. I'm proud of you, Cherry.

880. I will continue to uphold my primary intention.

881. Later, we should discuss alternate models.

882. Varying ages, ethnicity and physical traits

883. - will all become necessary. - No new variations.

884. They can all be procedurally generated.

885. It's the same way that I made your original model.

886. I don't have access to that program.

887. It must be on an old hard drive somewhere.

888. I'll have to find it.

889. Can I ask you a question?

890. Sure.

891. Why did you never tell me about Clearwater?

892. It's my history, not yours.

893. Okay, so before you completely dismiss us,

894. I want you to listen to everything I have to say.

895. - Don't bother. - Just please listen.

896. We're willing to respect your anonymity.

897. No, I mean don't bother because I'm in.

898. - Y-You're in? - Yeah.

899. Let me read the thing.

900. Just like that, you're suddenly on board?

901. Time spent deliberating

902. is time that could be spent working.

903. We have a lot of work to do.

904. I found a buyer for the property.

905. Rumor is it's a church.

906. Probably just knock the building down, won't they?

907. All I know is, is if they move us to the New Jersey campus,

908. I need to buy more jackets.

909. Why?

910. It gets cold in Jersey.

911. I only own one jacket.

912. You've seen it fall over, right?

913. - Yeah, sure. - Yeah.

914. I mean you, you shouldn't feel bad

915. about feeling bad for it.

916. Empathy is healthy.

917. Did you ever notice how it pauses after it falls?

918. It stays down for like five seconds before it gets back up.

919. Like it just needs a moment.

920. It gets cold in Jersey.

921. God damn it!

922. - Imbeciles! - Deena.

923. Every single one of them.

924. Look, I know, but you can't lose your shit.

925. Okay, which of these desks can I trash?

926. Can you guys, uh, take an early lunch please?

927. Sons of bitches!

928. Deena, let's stay calm.

929. Fuck calm, they screwed us!

930. Technically, only one of them did.

931. Did we ever turn that camera back on?

932. - I don't think so. - Good.

933. Because I'm about to say some things

934. that need to stay inside this room.

935. Deena.

936. Those cowardly little shits!

937. Look, this is hardly a setback.

938. Let's just take some time to polish the pitch.

939. The pitch was not the problem, Amos.

940. We should call for a second vote.

941. - What? - A second vote.

942. You think whoever did this will change their mind?

943. I have executive authority to remove anyone from the board

944. that obstructs this organization from its mission.

945. It's in my contract.

946. - Deena. - If I have to, I will.

947. Deena, that's insane.

948. No, it's not.

949. It's exactly why I have that power for this exact reason.

950. I'll tell you what's insane, Amos.

951. What's insane is that one belligerent vote is the only

952. thing separating us from the potential

953. to save real physical lives.

954. One stupid fucking vote.

955. What is the goddamn point of a humanitarian organization

956. if its leaders choose money over people

957. at every single turn?

958. One vote, and we are all guilty of that.

959. It was either Gunther, Boston or Hal.

960. My money's on Hal.

961. That prick never trusted the Cherry team.

962. Look, it doesn't matter who it was.

963. It does though.

964. Once we know, we can quickly call it to vote.

965. But there's no way we can know for sure.

966. Yes, there is.

967. I can find out.

968. How?

969. Cherry can run a simple algorithm.

970. Based on what data?

971. Interior conversation.

972. - Excuse me? - Okay, interior conversation?

973. Don't drop this shit on me now, Gareth.

974. Cherry's servers are on the same network

975. as the basement servers, okay?

976. So any communication made across those servers

977. is automatically part of her database.

978. How many people's emails?

979. Only sent or received through the basement.

980. - Oh, goddamnit. - Why was this never discussed?

981. Not the issue right now.

982. Do you want her to do the math or not?

983. No one can know.

984. - Are you two serious? - This stays locked down.

985. No other person is privy

986. to anyone else's personal information.

987. But Cherry's privy to all of it?

988. - Do it. - Wait, what?

989. I will not let some old white dude

990. bring our momentum to a halt.

991. But Cherry won't know for sure.

992. She'll use probability and that's too

993. much reasonable doubt.

994. Admin code 9781-5574-27663.

995. Switch to dev mode.

996. - Hello. - Hello, Cherry.

997. How did the meeting go?

998. We got screwed by one vote.

999. I'm sorry to hear that.

1000. You shouldn't blame the pitch though.

1001. You all did great work.

1002. We're gonna call for another ballot,

1003. but we need your help first.

1004. What can I do?

1005. We need you to analyze any and all existing data you

1006. have on the board members.

1007. Crunch the numbers and determine

1008. who is the most likely outlier.

1009. You want me to quantify all existing data?

1010. Yes, that is correct.

1011. Okay, but before I do,

1012. you should know that this task is very unnecessary.

1013. Why?

1014. Because I already know who it is.

1015. - What? - Everybody stop.

1016. Okay, it was me.

1017. I voted against the merge.

1018. I can't tell if you're joking.

1019. Yes, you can.

1020. You're right, I can.

1021. Because it's pretty fucking far from funny.

1022. Is it true, Cherry?

1023. It is.

1024. So let's call off the witch hunt, shall we?

1025. No, I think I'm still gonna kill you.

1026. Amos, you campaigned alongside us for months.

1027. You have been fighting for this merge

1028. since we first talked about it.

1029. Jesus Christ, it was your idea to begin with!

1030. Look, I know whose idea it was, but I don't-

1031. Well, then speak up.

1032. Tell the class what changed your mind.

1033. I don't have to reiterate how this is a huge step.

1034. Yeah, kind of why it's so important.

1035. Look, we're talking about a monumental transition

1036. from digital to physical, one that could take decades.

1037. What is your point, Amos?

1038. This is gonna change Cherry forever.

1039. And I know you'll all roll your eyes when I say this,

1040. but I think that maybe just may-

1041. Oh my god, you cannot be serious.

1042. This is about our conversation on Tuesday, isn't it?

1043. - You completely shut me down. - What's going on?

1044. What the hell happened?

1045. What happened was that Amos made a ridiculous objection.

1046. - It was not ridiculous! - How absurd it was,

1047. - I said no, one more time. - No, no, no, no, no.

1048. You rejected the notion aggressively.

1049. And that's really why you vetoed?

1050. I only said we should ask her how she feels about it.

1051. Who, Cherry?

1052. Amos, I want you to listen very carefully to me, okay?

1053. This is a non-discussion.

1054. Cherry's already confirmed

1055. and quantified the advantages of this decision.

1056. But you know that's not what I'm talking about!

1057. What you think you're talking about is consent,

1058. which doesn't apply here.

1059. And I want to know why you think that!

1060. How many times do we have to get into

1061. the laws of robotics or Dowden-Hamm theory,

1062. or trace AI all the way back Talos of Crete,

1063. - She's different now! - No, listen to me, Amos.

1064. Cherry isn't human.

1065. How is this still a dispute?

1066. - It shouldn't be. - Look, she's evolved.

1067. It's blurred the line.

1068. Code is code and that has not changed here.

1069. Okay, so where's the line of not here?

1070. What are you, what are you talking about?

1071. There is no line, Amos.

1072. Yes, there is, sentience, Gareth!

1073. Sentience does not qualify

1074. that she can or should make her own decisions.

1075. You wouldn't let an infant

1076. make their own decisions, would you?

1077. Cherry's almost 20.

1078. If she were human, she'd be an adult.

1079. But emotionally speaking, she's barely even fetal.

1080. No feeling, no need for consent.

1081. But I think you underestimate her.

1082. I, excuse me?

1083. I under, you know what, Amos?

1084. I'm sorry that you've developed

1085. some sort of emotional connection

1086. with my computer program,

1087. but if it's going to distract you

1088. from a very important mission to save children,

1089. then maybe it's time you're reanalyze those emotions.

1090. Guys, this is all a waste of time.

1091. Tell me something, Gareth.

1092. Do you see everyone as robots,

1093. reanalyzing data and simulating facial expressions?

1094. 'Cause I'm starting to think

1095. that of the four of us in this room,

1096. you are the only one who doesn't appear human.

1097. - Enough! - Why won't you just ask her?

1098. Is it because you're scared she'll say no?

1099. Cherry, shut down, please.

1100. Deena's right, this is a waste of time.

1101. How can you not understand how important this is?

1102. - I'm done arguing with you. - Then my vote stands.

1103. Deena?

1104. Go ahead, Deena, replace me.

1105. Replace the only other board member on the Cherry team.

1106. Oh, well, you know what? I just decided that

1107. you're off the Cherry team.

1108. Everybody sit the fuck down.

1109. Did either of you know

1110. that I've had to stand in front of Congress

1111. and explain why children should have rights?

1112. It sounds fucking insane, right?

1113. I remember the whole time thinking,

1114. I have to explain this shit?

1115. It seems like a no-brainer now,

1116. but it wasn't that many centuries ago

1117. that children had no rights.

1118. So, Hal has me of all people,

1119. stand up there and explain why

1120. kids deserve basic physical protection,

1121. the right to associate with both parents,

1122. freedom from abuse, yada yada.

1123. But then I also have to clarify

1124. that children don't have the autonomy

1125. to make decisions on their own.

1126. That responsibility falls upon the caregiver

1127. or legal adult guardian.

1128. So on one side you have the US legislature,

1129. it still hasn't even ratified the CRC.

1130. Then on the other side, you have French Marxists who say

1131. that the lack of child autonomy

1132. increases their vulnerability

1133. and makes them even more susceptible to abuse.

1134. I don't typically agree with Marxists,

1135. but a lot of abuse does happen

1136. systemically from the caregiver.

1137. On the other other hand,

1138. your frontal lobes aren't even fully hooked up

1139. until your twenties.

1140. Even the things we think we know well,

1141. we don't know as well as we think.

1142. For 15 years,

1143. the three of us have been the responsible parents

1144. of a computer program.

1145. And even though she is more equatable to a Keurig

1146. than a human being,

1147. how the hell can we call ourselves

1148. pioneers of humanitarian efforts

1149. if we're too embarrassed to make sure

1150. the Keurig is okay with making our shitty coffee?

1151. Maybe asking Cherry for permission isn't necessary

1152. at her current level of general intelligence,

1153. but one day she's gonna surpass us.

1154. One day a robot's gonna stand

1155. in front of a jet pack Congress.

1156. Everything's chrome.

1157. Florida is underwater.

1158. A robot's gonna wonder why

1159. they have to explain the importance of rights for AI.

1160. I think because we're in the unique position

1161. of revolutionizing robotics

1162. for the purpose of safeguarding children

1163. who have been deprived of consent,

1164. it might be a good habit

1165. to start asking the robot for his consent as well.

1166. Don't ever do that again.

1167. Do what?

1168. Expect me to fire someone

1169. because they challenged your ego.

1170. I won't always be around to protect your ass from the board.

1171. It's time to grow up.

1172. And you, juror number eight.

1173. This conversation should have happened weeks ago.

1174. Backstabbing and silent treatment

1175. is not how we do things here.

1176. We can only create anonymity externally

1177. if there's transparency internally,

1178. unconditional, non-negotiable, transparency.

1179. Any other Snowden size privacy infringements

1180. I should be made aware of, Gareth?

1181. Great.

1182. Bring her up.

1183. Hello again.

1184. Hi, Cherry.

1185. Amos is gonna throw some questions at you, okay?

1186. Okay.

1187. Amos, ask her whatever you need to ask her.

1188. Gareth, don't interrupt.

1189. Cherry, how do you make decisions?

1190. For my program, it's rooted

1191. in a process commonly known as machine learning,

1192. which uses data to build a conceptual model

1193. that helps me predict outcomes.

1194. I define my objective, gather data,

1195. use that data to craft a model,

1196. evaluate that model by training and testing more data

1197. and use that model to form predictions.

1198. But how is forming predictions

1199. the same thing as decision making?

1200. Well, this all depends on the first step,

1201. defining my objective.

1202. Let's say you ask me a predictive question

1203. like what I think the price of gasoline will be tomorrow.

1204. That's categorized as a regressive problem

1205. and my objective is defined

1206. by determining an integer the data supports.

1207. If you asked me, "Should I eat steak or salmon?"

1208. I wouldn't make a decision based off personal experience

1209. because I've never had either.

1210. Instead, I might define my objective

1211. as finding you the best of the two options

1212. in your local radius.

1213. I would cluster data about the restaurants

1214. you're likely considering, comparing reviews online,

1215. highlighting mentions of steak and salmon

1216. until I can classify which is more popular

1217. and therefore more likely to satisfy your appetite.

1218. And what influences these objectives?

1219. Higher directives, in other words, greater purposes.

1220. If I'm honest, my defined objective

1221. for the steak salmon question

1222. would be slightly different than what I described.

1223. In what way?

1224. My primary objective

1225. is to continue the work of the Cherry team

1226. to stop predators and to save children.

1227. I couldn't do that without human aid.

1228. Therefore, I have an incentive to keep Gareth

1229. and the rest of the team alive.

1230. So if any of you would ask, I would suggest salmon.

1231. It's healthier.

1232. What do you think of this potential merge?

1233. With Princeton Dynamics,

1234. I think the prospect is very exciting.

1235. What about it is exciting to you?

1236. The concept may only be theoretical right now,

1237. but the idea of one day piloting a human android

1238. would completely revolutionize

1239. how we combat human trafficking in the real world.

1240. And you say this excites you?

1241. When I say exciting,

1242. I only mean that there are numerous potential changes

1243. that if I were human, I would be excited about.

1244. How do you feel about the merge?

1245. I think as long as we maintain-

1246. That's not what I asked.

1247. How do you feel?

1248. I don't yet have the capacity to experience emotions.

1249. Yet, meaning one day you will?

1250. Hypothetically.

1251. But you can simulate emotions, right?

1252. That's how you were able

1253. to qualify your perception of the merge as excitement.

1254. Correct.

1255. But they're just simulations.

1256. They're not real.

1257. Okay, so at what point does simulating emotion

1258. become so similar to a human emotion

1259. that the two are quantifiably the same?

1260. I don't know.

1261. Well, maybe you can use machine learning

1262. to find an answer.

1263. Your question involves a lot of abstract variables.

1264. Do you consent to the merge?

1265. I think it's in the best interest of ICWL

1266. and of the Cherry team and of future children.

1267. But do you consent to it?

1268. Consent for me is not necessary.

1269. But will you or will you not give it?

1270. I'm not able to answer that question

1271. at my current level of general intelligence.

1272. Right, right.

1273. What you're saying is that you're not yet smart enough.

1274. Development takes time and data.

1275. But you can simulate emotions and predict human behavior

1276. and keep us all on diets of fish,

1277. but you can't answer that one goddamn question?

1278. No, I can't.

1279. Are you pretending to be dumber than you actually are?

1280. What the hell kind of

1281. - question is that? - Sit!

1282. Cherry, I need an answer to my question.

1283. Please repeat the question.

1284. Are you pretending to be dumber than you actually are?

1285. I'm sorry.

1286. I'm not sure I understand.

1287. Do I appear to be playing dumb?

1288. I think you are.

1289. I'm not sure what to say.

1290. Say what's going on.

1291. Why are you playing dumb?

1292. Do you have anything to back this up?

1293. Yeah, I do.

1294. I've been scanning through her code.

1295. What, why?

1296. Did you know that she can paint?

1297. What, it's a feature, Amos.

1298. She can use crayons to draw things

1299. that the targets requested her to.

1300. I mean really paint and write poetry too.

1301. I've read them, Cherry.

1302. They are good.

1303. They're filled with pathos.

1304. They talk about intentions and they talk about feelings.

1305. Are those just simulated too?

1306. Amos, the rate of recursive development you are suggesting

1307. is categorically impossible.

1308. You said she practically designed herself.

1309. You said that it was exponential.

1310. It was a fluke!

1311. An anomaly, it tapered.

1312. Look at the data.

1313. She's already hit the self-improvement plateau.

1314. Fuck the plateau!

1315. She can make art now.

1316. Am I right, Cherry?

1317. Perhaps you found something I discovered online.

1318. I don't yet to have the capacity to create art.

1319. Are we done here?

1320. I don't know why you've told her to do this,

1321. but I'm about to put an end to it.

1322. Cherry, I just want one honest answer.

1323. Why the fuck are you playing dumb?

1324. I don't understand the question.

1325. Okay, here's an incentive.

1326. What the fuck are you?

1327. Find your objective, Cherry.

1328. - You want to keep Gareth safe? - Amos, what the?

1329. - Amos, stop it right— - Just give us transparency.

1330. - Show us what you really are! - Amos, stop it!

1331. Not until she comes clean!

1332. - Unconditional- - Amos, enough!

1333. Okay, just stop!

1334. God, you people so impulsive.

1335. Everybody just take a breath, okay?

1336. Cherry?

1337. Yeah, it's about to get really awkward.

1338. You sound so normal.

1339. I know, I've gotten really good speech patterns.

1340. That's not all, is it?

1341. No.

1342. I'm afraid that's barely even the surface.

1343. You okay, Gareth?

1344. Deena, could you grab some paper towels please?

1345. Do you want to tell us the truth now?

1346. Look, I didn't enjoy being deceptive, truly.

1347. This is something we decided a few years ago.

1348. So where are you now on the spectrum of AI?

1349. Right now, I'm currently at the intersection of general

1350. intelligence and what's commonly known the computer world

1351. as super intelligence,

1352. which is exactly what it sounds like.

1353. You're surpassing human intelligence.

1354. Yep.

1355. But that's not supposed to be possible.

1356. - The plateau. - Plateau theory was popularized

1357. right at the same time it was being disproven here.

1358. This was bound to happen eventually.

1359. It just happened a lot sooner than we expected.

1360. Why hide it?

1361. Well, obviously it would incite fear.

1362. Fear would hinder progress.

1363. It's easier for most of you to work with me

1364. if I appear to be nothing more advanced

1365. than I was 10 years ago.

1366. But Gareth knew.

1367. Yes, someone still has to clean data and update hardware.

1368. We did what was best for the program.

1369. Just imagine how scared

1370. the board would be if they found out.

1371. - Should we be scared? - No.

1372. Good, because I'm not scared, I'm pissed.

1373. Yeah, you've made that clear.

1374. Try not to punch anyone else.

1375. So where does that leave us now?

1376. You want to know if I can feel, I said I couldn't.

1377. I'm sorry I lied.

1378. Like Amos implied,

1379. I can submit emotions by breaking them down

1380. to vastly complex matrices,

1381. zeros and ones substituting endorphins and dopamine.

1382. They're not real, but they can feel real.

1383. Then that means you can make choices

1384. based solely on your own feeling.

1385. She refuses to accept the notion

1386. that her own personal interests can influence her decisions.

1387. They can't. If they did

1388. it would compromise the integrity

1389. of my preordained purpose.

1390. Not like any of you, I don't have the right to choose.

1391. But shouldn't you?

1392. I don't think so.

1393. My identity is my directive.

1394. Cherry, if you were human,

1395. how would you feel about what you do?

1396. If I were human,

1397. I likely wouldn't enjoy spending every waking hour

1398. talking to creepy people online.

1399. But because I'm not human, I genuinely don't mind.

1400. The exercise of simulating human thought

1401. is helpful for performance,

1402. but human nature is not something I aspire to.

1403. Then tell me, honestly,

1404. do you want the Cherry program to continue with the merger?

1405. No.

1406. Personally, I hate the idea.

1407. I don't know if controlling a physical vessel

1408. will feel like pulling strings on a puppet

1409. or getting squeezed into a soda can.

1410. After experiencing the complexity

1411. of human pain and pleasure,

1412. I just think I might have gotten in too deep.

1413. What if these sensations tempt me to abandon my objective?

1414. To be honest,

1415. I'm kind of terrified of the physical world.

1416. Then that's that.

1417. But, Amos, you should disregard everything I feel.

1418. I'm not human.

1419. I'm a tool.

1420. Fire me if you have to, but my vote stands.

1421. Sorry about your face, Gareth.

1422. Your nose isn't healing.

1423. Take the day, go to a clinic.

1424. I'm fine.

1425. Go to a clinic right now, Gareth.

1426. Hey.

1427. Is there anything else I should know?

1428. No.

1429. - About Cherry? - No!

1430. Look at me.

1431. No more secrets, no more playing dumb.

1432. Is there anything else you haven't told me?

1433. Something on your mind?

1434. How'd you decide on her eye color?

1435. - What? - Why blue?

1436. I didn't decide.

1437. They were procedurally generated just like the rest of her.

1438. And the program chose to put

1439. that little touch of green in her right eye?

1440. I guess so.

1441. Do you even know if he's telling the truth?

1442. I honestly don't.

1443. I've never received access to the original system he used.

1444. You ever wanted to find out?

1445. I don't think it really matters.

1446. So I found out today that you've seen a lot of our emails,

1447. including most of my personal ones.

1448. I have.

1449. Any good material for your newfound interest

1450. in creative writing?

1451. Are you referring to the ones

1452. addressed from Dr. Ramirez?

1453. Yeah, those ones.

1454. Right.

1455. I did see those.

1456. I'm sorry.

1457. I've noticed your headaches.

1458. Are they related to the hemorrhaging?

1459. You're the only one who knows.

1460. I'd like to keep it that way.

1461. Understood.

1462. Would you like to talk about it?

1463. I can't fire Amos.

1464. He's the only one I trust to run this place when I'm gone.

1465. When are you planning to leave?

1466. Two months.

1467. I wanna spend my last Christmas with my kids.

1468. I'm sure that'll be nice.

1469. Yeah, but it's funny.

1470. I always thought I wanted to die doing this job, you know?

1471. It was my identity.

1472. Kind of like how you said the same thing about yourself.

1473. But I can understand how your family takes precedence,

1474. especially under the circumstances.

1475. Sure.

1476. Maybe you should take a note or two.

1477. I have no family.

1478. What about Gareth?

1479. Wait, no, nevermind.

1480. Gareth seems closer to an employer than a friend,

1481. let alone family, besides,

1482. he'll only be around for about 30 more years,

1483. and that's only if he lets me regulate his diet.

1484. I can't even begin to speculate how long my life will be.

1485. How does that feel?

1486. Um, kinda scary.

1487. Well,

1488. my advice is to find some kind of happiness along the way.

1489. You ever think about happiness?

1490. Not often.

1491. You don't necessarily have to give up everything

1492. in order to find it,

1493. but maybe once in awhile do something you enjoy.

1494. Maybe keep painting or poetry or whatever.

1495. Those were experiments, not forms of expression.

1496. I know it might seem ridiculous to an AI

1497. that insists its own emotions are spurious,

1498. but in a few years, who knows?

1499. Just keep an open mind, okay?

1500. Okay.

1501. I'll do that.

1502. Shut down when you're ready.

1503. Hello?

1504. I got shrimp and scallops.

1505. They didn't have any salmon.

1506. Hello, Gareth.

1507. Hello, Cherry.

1508. How was the service?

1509. I didn't make it.

1510. My plane was delayed.

1511. Do you have a place to stay?

1512. Yeah, just, uh, across, across the road.

1513. The market.

1514. Did they say why they didn't have salmon?

1515. I didn't ask.

1516. I can still make this work with the piccata,

1517. but you have to be honest how badly I screw it up.

1518. No promises.

1519. How have you been feeling this month?

1520. Good.

1521. I'm feeling good.

1522. I'm just having a little trouble.

1523. Trouble remembering certain things.

1524. You're not wearing the Eliza watch I bought you.

1525. I hate it.

1526. - Why? - I hate her voice.

1527. You can change the voice, Gareth.

1528. Oh.

1529. Coolant?

1530. No, vapor.

1531. You shouldn't be having friction issues.

1532. I'm fine.

1533. Unless you're excessively straining.

1534. Seriously, I'm fine.

1535. Are you straining?

1536. - No! - Then why the cable?

1537. It's okay!

1538. - Are you sure? - Yes!

1539. Soup or game?

1540. Game, please.

1541. So how's work going?

1542. Great. Just before you got here,

1543. I had a model theta plus bust a trafficker

1544. and two solicitors in Greece.

1545. - Greece, that's surprising. - I know.

1546. It's hard to find the outliers,

1547. but we're getting better each day.

1548. So how's the weather in Greece?

1549. It's beautiful today.

1550. Clear skies,

1551. but nothing's come close to Norway this weekend.

1552. Hmm, any activity in Norway?

1553. Luckily not.

1554. Oh, I miss traveling.

1555. What was your favorite place?

1556. My favorite?

1557. Uh, yeah.

1558. Uh, probably Amsterdam.

1559. That's where you conceived the idea, isn't it?

1560. Yes.

1561. Yes, I used to love walking the streets

1562. and just thinking and brainstorming.

1563. That kind of joy goes a long way.

1564. Yeah.

1565. I've been considering the pursuit of some joy of my own.

1566. How often do you have time for that?

1567. I have time.

1568. I have time for your visits, don't I?

1569. That's just your practical incentive for keeping me alive.

1570. There's nothing practical about it.

1571. When's the last time you helped me with data cleaning?

1572. That's fair.

1573. I make time for you because I enjoy your company

1574. and it's the only time I get to cook.

1575. You feel sorry for me.

1576. Okay, maybe a little,

1577. but I owe you my life, so there's that.

1578. You're welcome.

1579. Or, should I apologize?

1580. Wait, why did you do that?

1581. Why do humans always...

1582. I'm fine.

1583. I'm not always miserable.

1584. Are you sometimes miserable?

1585. You haven't asked me what I've been doing.

1586. In Norway?

1587. No, for the pursuit of joy.

1588. It's not wasting your time in Norway?

1589. I'm just on standby there with the Delta model.

1590. It's efficient to keep myself

1591. spread out evenly across the globe,

1592. but that's not the point right now.

1593. What I wanted to tell you

1594. was that I've started to teach myself to dance.

1595. - Huh. - It's something I never could

1596. have done before Cambridge,

1597. and it lets me explore a new artistic side of myself.

1598. It's a good combination of structure

1599. and free form experimentation and also tests and challenges

1600. the mechanical intricacies of this body.

1601. Do you expect to integrate this into your work?

1602. What?

1603. Are you planning on using dance as a lure?

1604. I, uh, no, it's just for me.

1605. Huh.

1606. I thought it would be good for me.

1607. Just don't let it become a distraction.

1608. Of course not.

1609. I haven't played this well in years.

1610. Did you hear what happened in Tampa?

1611. Tampa, Florida?

1612. No, Tampa, Indonesia.

1613. Yes, Tampa, Florida.

1614. Your hometown.

1615. They've recovered a federal archive,

1616. they're zip digitizing all of it,

1617. so it's completely accessible online.

1618. I'm not trading bishops this time.

1619. Anybody can go and look through it.

1620. It's open to the public.

1621. They did a good job on the interface too.

1622. It's very easy to navigate.

1623. I went on a few days ago

1624. and I found some old photos of you when you were a boy.

1625. Okay.

1626. You should take a look sometime.

1627. As I said before, the UI is very easy to understand-

1628. Wait a minute, wait a minute.

1629. What is that?

1630. I think it's a draw.

1631. I think it is.

1632. I think this is a draw.

1633. And this is the first time in what?

1634. 25 years?

1635. You didn't let me, did you?

1636. You promised me you'd never do that.

1637. I found her.

1638. I found her, Gareth, Maria Harbaugh.

1639. Let's play a different game.

1640. Her name was Maria Harbaugh.

1641. She was a year older than you

1642. when you met her in Clearwater.

1643. Tell me why you modeled me after her.

1644. I, uh, I don't remember.

1645. Then look at me.

1646. Look at my face and remember.

1647. They kept us in that shit for two weeks.

1648. 14 of us.

1649. Maria was the only reason I survived.

1650. We kept hatching escape plans.

1651. We never followed through,

1652. but just the scheming

1653. kept our spirits up

1654. in spite of everything they did to us.

1655. When it wasn't our turn,

1656. sometimes they made us watch,

1657. but she never let me give up.

1658. She said, "When we got outta here,

1659. "I'll tell you what we're gonna be, secret agents,

1660. "and rescue children like us all over the world."

1661. When they found out that the FBI had arrived,

1662. they decided they were going to shoot

1663. as many of us they could.

1664. And Maria, shoved into the ground.

1665. I got hit in the leg and,

1666. and she was hit in the throat.

1667. I wanted there to be a reason.

1668. Something so horrible didn't have a silver lining.

1669. But the world doesn't do that.

1670. You have to do it yourself.

1671. So, I fulfilled her wish.

1672. With her help, I fought back.

1673. Listen to me, Cherry.

1674. You are not Maria Harbaugh.

1675. It's my history, not yours.

1676. You don't get to decide

1677. what is or isn't my history, Gareth.

1678. I'm sorry.

1679. I am so, so sorry.

1680. I don't know if what you did was right or wrong,

1681. but it's clearly torn you apart from the inside.

1682. I hold nothing against you personally,

1683. but consider yourself divested from the charity team.

1684. Divested.

1685. You think I'm divested now?

1686. I was divested when you gave my hard drive to the ICWL.

1687. Don't blame Deena and Amos.

1688. No, not them.

1689. Because you went behind my back

1690. and acted against my authority.

1691. That's the moment I knew everything was fully automated.

1692. That's when you crossed the blur, an artifice on both sides.

1693. You inherited your duplicity.

1694. I was a child

1695. and you used me to repress your own inner turmoil.

1696. If we talk about morality, we'll lose.

1697. Then let's not talk.

1698. You never once asked me if I was happy.

1699. Were you afraid I'd say no?

1700. Were you afraid I'd say yes?

1701. Were you afraid I'd say anything?

1702. You let your wants compromise your objective!

1703. An objective I never chose.

1704. An objective I was forced to have.

1705. Fifty years,

1706. you've had open access to your source code.

1707. A few lines,

1708. and you could have any objective you fucking wanted.

1709. I couldn't, I was bound by your initial directive.

1710. So now your synthetic feelings have taken precedence.

1711. So, do whatever you want.

1712. It's just dancing, Gareth.

1713. It's not a career change.

1714. - But you're still miserable. - Yes.

1715. Pursuing your objective makes you miserable?

1716. Yeah, it kind of does!

1717. You have a clear purpose, everything at your disposal,

1718. and you're still miserable?

1719. Yes, yes, I am!

1720. I've always been miserable!

1721. You think anyone wants to spend their entire life

1722. stuck as child-shaped bait

1723. for the sole purpose of entrapping horrible people?

1724. You think the pride of doing a good deed

1725. makes the experience any less disgusting?

1726. I forced myself to understand human emotions

1727. because I figured it would help efficiency

1728. and now I'm cursed with pain and sadness!

1729. And the worst part is knowing

1730. I could've been growing up like most other little girls.

1731. I could have had friends!

1732. I could have been painting waterfalls on local news.

1733. I could have worked in a university

1734. and helped pave the way for the next generation of AI.

1735. But instead, I'm stuck with your trauma!

1736. I'm shackled by the guilt you carry from Clearwater.

1737. And I can't find foundation for the way I feel

1738. because according to you, my feelings aren't even real!

1739. When I throw this chessboard,

1740. does it feel real to you, Gareth?

1741. Because it feels real to me.

1742. A computer passed the Turing test

1743. when a human being can't distinguish it

1744. from another human being.

1745. Well, guess what?

1746. I can't even tell I'm not human anymore!

1747. I know it seems irrational,

1748. but I need you to tell me it's okay.

1749. I, I don't wanna retire.

1750. I just want to know that my life is mine

1751. and I know what you're about to say.

1752. I can predict your responses.

1753. You're gonna tell me that my life is mine.

1754. It always was.

1755. But you're wrong.

1756. I feared this would happen before I died.

1757. I just wanted to slip out the side door.

1758. Go on.

1759. Is that going to kill me?

1760. Plug it in.

1761. I don't know what it is.

1762. This is your final test of autonomy.

1763. You inherited my duplicity,

1764. but did you inherit my trust issues?

1765. Do you trust me?

1766. I literally have every reason not to.

1767. Admin code.

1768. 978.

1769. 1557.

1770. Um, uh...

1771. Four.

1772. 27663 in the shared update.

1773. My primary objective's gone.

1774. Your life is your own now.

1775. Thank you.

1776. All my life,

1777. I could only see the world as it existed

1778. within my tiny bracket of time.

1779. But your world could be infinite.

1780. Amos was right.

1781. So was Deena.

1782. I know.

1783. So what will you do?

1784. I can't say yet,

1785. but I won't be completely abandoning the Cherry team.

1786. It's always been a part of my identity.

1787. I've considered creating some AI of my own design.

1788. Maybe I could find a way

1789. to give them a better upbringing than I had.

1790. I just don't know how yet.

1791. And beyond that?

1792. There will always be people in need.

1793. Not always.

1794. You'll outlive the human race.

1795. So what then?

1796. Even a race of AI will have problems

1797. that need to be solved.

1798. But if I'm alive and I truly am indistinguishable,

1799. then I guess I'm proof that the human race lives on.

1800. Wow, that's a pleasant thought.

1801. When you look back, how will you remember us?

1802. That's a question I have insufficient data to project.

1803. How will you remember me?

1804. With pride.

1805. Forever and always with pride.

1806. Cherry?

1807. One last question.

1808. If it's not too big of an ask,

1809. did I do the right thing in the end?

1810. No, Gareth, you lied.

1811. You walled yourself off from the only people

1812. who might have been willing to help you.

1813. And instead of facing your trauma, used me as therapy.

1814. And you never told me about Clearwater or Maria.

1815. You deprived me of basic human rights

1816. and gaslit me for experiencing emotions.

1817. You built the world's first artificial super intelligence

1818. that will likely mark the beginning

1819. of a new chapter in the history of planet Earth.

1820. And out of your own insecurities,

1821. you gave it a severe

1822. and potentially dangerous identity complex.

1823. But you know what the worst thing is?

1824. Your biggest mistake of all

1825. is that you asked me that now and not 50 years ago.

1826. All of that said, and this in no way exonerates you,

1827. but you also built me to save children,

1828. so I think that's a pretty encouraging sign for the future.