Aligning AI with AI-Commerce

https://www.youtube.com/watch?v=2HglwEL9Dtc&list=PL3C6eF-zu5AYohNL1ZgOBqlwwJ29x-lTO&index=33

speaker

• Don Gosson (Nevermind co-founder and CEO, Ocean Protocol co-founder) theme
• AI alignment through economic incentives among AI agents based on "AI Commerce

1. background and claims

• AI agents as new "consumers
  • In the future, a large number of AI agents will enter commerce, and not only human-to-human transactions but also AI-to-human transactions will increase. This is called "AI Commerce" and is expected to create a huge economic zone.
• Importance of payment infrastructure to take advantage of data and AI models
  • If payments between AI agents are held by a centralized service, "exclusion authority" could be concentrated on the platform side. Therefore, we believe that a decentralized payment and economic system is needed.

2. relation to AI alignment

• What is AI Alignment?
  • Efforts to match AI with the intentions, values, and ethics of the humans who use it. Currently, there are three main approaches: 1.
    • Incorporation of ethical guidelines (restrictions on inappropriate language, etc., in the model itself)
    • Adjustment by human feedback such as RLHF
    • Model constraints (e.g., stop at red light in automatic operation)
• A New Approach: Economic Alignment
  • In addition to the above, "economic penalties and rewards" for commercial transactions between agents may control and guide AI behavior.

3. examples of multi-agent systems and economic incentives

• Smallville Experiment (Stanford)
  • If 25 AI agents were given individual "personalities" and simulated a virtual world for two days, unexpected behavior (e.g., going to a bar for lunch and drinking from noon) would occur. If there is an economic penalty (e.g., being fired by an employer), the behavior might change.
• Specific Examples of Economic Incentives
  • Rewards: Payments to agents for desired actions, point rewards, etc.
  • Penalties: fines for inappropriate behavior, suspension of business, loss of reputation, etc.
  • Pros: easy to promote desired behavior
  • Cons: System complexity and potential for fraud and bias

4. actual cases/applicability

• Auto insurance premiums (e.g., Arizona x Tesla)
  • Policies aimed at lowering insurance premiums and other factors indirectly change AI driving behavior.
• Healthcare (U.S. HRRP)
  • AI algorithms for preventive management as patients are penalized for readmission.
• AI Content Generation and AI Trader
  • When AI works on a pay-for-performance or commission basis, its algorithms behave according to incentives.

5. challenges and future steps

• issue
  • Incentive design difficulties (fairness, effectiveness, technical implementation)
  • Unintended side effects (agents may look for loopholes and take competing actions)
  • Balancing short-term and long-term interests
  • Who makes and sets decisions (regulation and governance)
• Future Outlook
  • Expansion of commercially useful agents
  • Dissemination of payment and settlement functions that enable agents to transact
  • Creating a mechanism to leverage economic incentives in areas that are ultimately difficult for humans to directly manage.

6. Q&A

• Who determines incentives and penalties and how?
  • There is potential to learn from the crypto and decentralized governance cases, but there is no clear solution yet.
  • Bias of entities with large funds or influence is a concern.

Overall Summary

• Don Gosson predicts that "AI Commerce," in which AI agents conduct commerce with each other, will expand in the future, and argues that a decentralized payment infrastructure to support it is critical. Furthermore, while economic incentives and penalties can be an effective means of controlling AI behavior, their design is extremely complex, and many challenges remain, including biases and loopholes. A major theme for the future will be how to introduce economic mechanisms and align AI in the "desired direction" when the number of commercially active agents increases.

This page is auto-translated from [/nishio/Aligning AI with AI-Commerce](https://scrapbox.io/nishio/Aligning AI with AI-Commerce) using DeepL. If you looks something interesting but the auto-translated English is not good enough to understand it, feel free to let me know at @nishio_en. I'm very happy to spread my thought to non-Japanese readers.