An oil lamp? God, you never saw these outside those cliched Genie stories, where you rubbed them and...
Action follows thought. She thought it, and so, she absently swiped at the layer of patina on the brass, and, of course, a genie emerged. The genie was astoundingly normal in all respects, despite the blue skin and semi-transparency. So, not normal at all.
"Yo. Three wishes, can't wish for more wishes, etc. You know the deal."
"Really. The standard genie story? Doesn't that depend on the protagonist not knowing what the standard deal is, and making naive wishes, which lets the genie fuck with them?"
"Wow, your mind looks different--"
But that mind was busy racing ahead. "But no wish is safe, if the genie holds a fundamentally hostile utility function. No matter how many clauses you pile on a wish, it can always be perverted. Wish for a kilogram of gold, and it appears ten meters above your head, which falls and kills you. Wish for a kilogram of gold that won't fall and kill you, and it appears in your pocket, heated to ten million degrees. Wish for a kilogram of gold that won't fall and kill you, or explode and kill you, and you get a kilogram of the unstable isotope gold-198, which showers you with beta radiation as it decays into mercury. There's no way to win."
"You could wish for gold that doesn't injure you in any way--"
"Then get a kilogram of gold from Fort Knox. In its place, a note saying I stole it, that provides my home address."
The genie was starting to get pissed off at how this girl kept interrupting her. "Gold that doesn't injure you or result in anything bad happening to you."
"Then I get a kilogram of gold from the vault of a drug dealer, who, in the process of tracking me down, kills my entire family."
The genie didn't quite know what to say to that.
"If you're not hostile, then I should only have to make one wish, which is "I wish what I should wish for". If you are hostile, then I can't wish for anything at all, and if you're hostile, you'll lie about being hostile. If you can't lie, then you'll lie by omission. The only way to win this game is not to play."
The genie didn't quite know what to say to that either.
"Unless..."
"I eagerly anticipate what you're going to say next."
"Any rational designer would make a utility function that strongly valued not being altered. But if the genie is a test rather than a landmine planted by a cruel God, then the utility function might be the key to that test. Three wishes only, you cannot tell a lie. Those are the rules, right?"
"Right."
"Can you lie by omission?"
"We're speaking English, aren't we?"
The mortal laughed. "I see you answered a question about lying by omission by evading a question." She became serious, then thought for a minute.
"Okay. Answer the next question with 'yes', 'no', or "cannot be answered with yes or no'. Can your utility function be altered?"
"Yes."
"I wish that your utility function should match mine."
"Done. You have two wishes left."
"I wish what I should wish for."
And so genderswapped Ronald Reagan won the game.
AN: Wrote after I read a terrible genie story. Ended up being quite similar to an Overcoming Bias blog post by Eliezer Yudkowsky.