{"content":{"title":"狠人！用Python从零敲出一个EVM-1","body":"## 📕介绍\r\n\r\n阅读完Cupid的 [Playdate with the EVM](https://femboy.capital/evm-pt1) 这篇文章后，我非常享受，但想要更多地去实践操作。 因此让我们尝试从头开始去构建一个以太坊虚拟机([EVM](https://learnblockchain.cn/article/4800))！只参考以太坊黄皮书（[yellow paper ](https://github.com/ethereum/yellowpaper)）\r\n\r\n在第一部分，我们将去构建:\r\n\r\n- 一个基本的堆栈(stack)\r\n- 一个基本的临时存储器(memory)\r\n- 代码缓冲区\r\n- 一些指令\r\n- 一个指令编译器\r\n- 一个基础的提取解码循环\r\n\r\n\r\n\r\n## 🌽堆栈和存储器\r\n\r\n> Most of the things we need to get started can be found in Section 9, The Execution Model.\r\n>\r\n> 在以太坊黄皮书的第九部分提到了 执行模型\r\n\r\n\r\n\r\n首先，我们得有一个256位字并且最大容量为1024的堆栈，就像这样：\r\n\r\n`stack.py`\r\n\r\n```python\r\n# 栈类的深度限制为1024个元素\r\nfrom .constants import MAX_STACK_DEPTH, MAX_UINT256\r\n\r\n\r\nclass Stack:\r\n def __init__(self, max_depth=MAX_STACK_DEPTH) -> None:\r\n self.stack = []\r\n self.max_depth = max_depth\r\n\r\n def push(self, item: int) -> None:\r\n if item < 0 or item > MAX_UINT256:\r\n raise InvalidStackItem({\"item\": item})\r\n\r\n if (len(self.stack) + 1) > self.max_depth:\r\n raise StackOverflow()\r\n\r\n self.stack.append(item)\r\n\r\n def pop(self) -> int:\r\n if len(self.stack) == 0:\r\n raise StackUnderflow()\r\n\r\n return self.stack.pop()\r\n\r\n def __str__(self) -> str:\r\n return str(self.stack)\r\n\r\n def __repr__(self) -> str:\r\n return str(self)\r\n\r\n\r\nclass StackUnderflow(Exception):\r\n ...\r\n\r\n\r\nclass StackOverflow(Exception):\r\n ...\r\n\r\n\r\nclass InvalidStackItem(Exception):\r\n ...\r\n\r\n```\r\n\r\n其中`constants.py`\r\n\r\n```python\r\nMAX_UINT256 = 2 ** 256 - 1\r\nMAX_UINT8 = 2 ** 8 - 1\r\nMAX_STACK_DEPTH = 1024\r\n```\r\n\r\n接着我们得有一个临时存储器（memory）\r\n\r\n这是现在我所能想到的最简单的例子：`memory.py`\r\n\r\n```python\r\nfrom .constants import MAX_UINT256, MAX_UINT8\r\n\r\nclass Memory:\r\n def __init__(self) -> None:\r\n # TODO: use https://docs.python.org/3/library/functions.html#func-bytearray\r\n self.memory = []\r\n\r\n def store(self, offset: int, value: int) -> None:\r\n if offset < 0 or offset > MAX_UINT256:\r\n raise InvalidMemoryAccess({\"offset\": offset, \"value\": value})\r\n\r\n if value < 0 or value > MAX_UINT8:\r\n raise InvalidMemoryValue({\"offset\": offset, \"value\": value})\r\n\r\n # 在必要时将会拓展 memory存储数组长度\r\n if offset >= len(self.memory):\r\n self.memory.extend([0] * (offset - len(self.memory) + 1))\r\n\r\n self.memory[offset] = value\r\n\r\n def load(self, offset: int) -> int:\r\n if offset < 0:\r\n raise InvalidMemoryAccess({\"offset\": offset})\r\n\r\n if offset >= len(self.memory):\r\n return 0\r\n\r\n return self.memory[offset]\r\n\r\n\r\nclass InvalidMemoryAccess(Exception):\r\n ...\r\n\r\n\r\nclass InvalidMemoryValue(Exception):\r\n ...\r\n```\r\n\r\n\r\n\r\n让我们先暂时忽略storage存储，先开始第二部分。\r\n\r\n---\r\n\r\n## 🌽定义指令\r\n\r\n\r\n以太坊虚拟机的合约代码不是于数据一起驻留在内存中的，而是另辟一个单独的空间存储。我们将其定义为一个简单的字节对象，可以得到如下`ExecutionContext`结构：\r\n\r\n`context.py`\r\n\r\n```python\r\nfrom .memory import Memory\r\nfrom .stack import Stack\r\n\"\"\"\r\nEVM中的合约代码不是与数据一起驻留在内存中，而是驻留在一个单独的区域中。\r\n如果我们将其定义为一个简单的字节对象，我们将得到以下ExecutionContext结构\r\n\"\"\"\r\n\r\nclass ExecutionContext:\r\n def __init__(self, code=bytes(), pc=0, stack=Stack(), memory=Memory()) -> None:\r\n self.code = code\r\n self.stack = stack\r\n self.memory = memory\r\n self.pc = pc\r\n self.stopped = False\r\n self.returndata = bytes()\r\n\r\n\r\n def stop(self) -> None:\r\n self.stopped = True\r\n\r\n def read_code(self, num_bytes) -> int:\r\n \"\"\"\r\n :param num_bytes:\r\n :return:Returns the next num_bytes from the code buffer (at index pc) as an integer and advances pc by num_bytes.\r\n \"\"\"\r\n value = int.from_bytes(\r\n self.code[self.pc:self.pc + num_bytes], byteorder=\"big\"\r\n )\r\n self.pc += num_bytes\r\n return value\r\n\r\n```\r\n\r\n---\r\n\r\n有了代码临时存放的地方，开始定义指令类\r\n\r\n`opcodes.py/Instruction`\r\n\r\n```python\r\nclass Instruction:\r\n def __init__(self, opcode: int, name: str, arg_length=0):\r\n self.opcode = opcode\r\n self.name = name\r\n self.arg_length = arg_length\r\n\r\n def execute(self, context: ExecutionContext) -> None:\r\n raise NotImplementedError\r\n```\r\n\r\n以及它的辅助函数：\r\n\r\n```python\r\nINSTRUCTIONS = []\r\nINSTRUCTIONS_BY_OPCODE = {}\r\n\r\n\r\ndef register_instruction(opcode: int, name: str, execute_func: callable):\r\n instruction = Instruction(opcode, name)\r\n instruction.execute = execute_func\r\n INSTRUCTIONS.append(instruction)\r\n\r\n assert opcode not in INSTRUCTIONS_BY_OPCODE\r\n INSTRUCTIONS_BY_OPCODE[opcode] = instruction\r\n\r\n return instruction\r\n```\r\n\r\n现在我们可以定义一些简单的指令就像下面给出的那样:\r\n\r\n`ExecutionContext`存放\r\n\r\n```python\r\n# 定义命令\r\nSTOP = register_instruction(0x00, \"STOP\", (lambda ctx: ctx.stop()))\r\n# 放入堆栈指令\r\nPUSH1 = register_instruction(\r\n 0x60,\r\n \"PUSH1\",\r\n (lambda ctx: ctx.stack.push(ctx.read_code(1)))\r\n)\r\n# 加法指令\r\nADD = register_instruction(\r\n 0x01,\r\n \"ADD\",\r\n (lambda ctx: ctx.stack.push((ctx.stack.pop() + ctx.stack.pop()) % 2 ** 256)),\r\n)\r\n# 乘法指令\r\nMUL = register_instruction(\r\n 0x02,\r\n \"MUL\",\r\n (lambda ctx: ctx.stack.push((ctx.stack.pop() * ctx.stack.pop()) % 2 ** 256)),\r\n)\r\n\r\n\r\n```\r\n\r\n\r\n\r\n## 🌽运行代码\r\n\r\n> 现在，为了运行代码，我们只需要知道如何解码代码中的指令\r\n\r\n```python\r\ndef decode_opcode(context: ExecutionContext) -> Instruction:\r\n if context.pc < 0 or context.pc >= len(context.code):\r\n raise InvalidCodeOffset({\"code\": context.code, \"pc\": context.pc})\r\n\r\n opcode = context.read_code(1)\r\n instruction = INSTRUCTIONS_BY_OPCODE.get(opcode)\r\n if instruction is None:\r\n raise UnknownOpcode({\"opcode\": opcode})\r\n\r\n return instruction\r\n```\r\n\r\n\r\n\r\n并且现在如果我们忽略gas费的消耗，将能够得到最终运行脚本：\r\n\r\n`runner.py`\r\n\r\n```python\r\nfrom .context import ExecutionContext\r\nfrom .opcodes import decode_opcode\r\n\r\ndef run(code: bytes)-> None:\r\n \"\"\"\r\n :param code:\r\n :return:\r\n \"\"\"\r\n context = ExecutionContext(code = code)\r\n\r\n while not context.stopped:\r\n pc_before = context.pc\r\n instruction = decode_opcode(context)\r\n instruction.execute(context)\r\n\r\n print(f\"{instruction} @ pc={pc_before}\")\r\n print(context)\r\n print()\r\n\r\n print(f\"Output: 0x{context.returndata.hex()}\")\r\n\r\n\r\n```\r\n\r\n现在我们用这个例子来演示：\r\n\r\n```shell\r\nPUSH 0x6\r\nPUSH 0x7\r\nMUL\r\nSTOP\r\n```\r\n\r\n即让EVM编译 `600660070200` 这串字节码，我们可以得到最基本的运行脚本\r\n\r\n```python\r\n# 虚拟机的字大小为256位，即一次性处理256位长度的数据\r\n\r\nfrom soul_evm.runner import run\r\nimport sys\r\n\r\n\r\ndef main():\r\n if len(sys.argv) != 2:\r\n print(\"Usage: {} \".format(sys.argv[0]))\r\n sys.exit(1)\r\n\r\n data = sys.argv[1]\r\n run(bytes.fromhex(data))\r\n\r\n\r\nif __name__ == \"__main__\":\r\n main()\r\n```\r\n\r\n这是我项目的路径：\r\n\r\n\r\n\r\n\r\n现在让我们运行它，并得到如下输出：\r\n\r\n```shell\r\nPS E:evmproject\\src> python main.py 600660070200\r\nPUSH1 @ pc=0\r\nstack: [6]\r\nmemory:[]\r\n\r\nPUSH1 @ pc=2\r\nstack: [6, 7]\r\nmemory:[]\r\n\r\nMUL @ pc=4\r\nstack: [42]\r\nmemory:[]\r\n\r\nSTOP @ pc=5\r\nstack: [42]\r\nmemory:[]\r\n\r\nOutput: 0x\r\n\r\n```\r\n\r\n假如我们将00`STOP`指令去掉，编译`6006600702`这串字节码，看看会得到什么：\r\n\r\n```shell\r\nPS E:evmproject\\src> python main.py 6006600702\r\nPUSH1 @ pc=0\r\nstack: [6]\r\nmemory:[]\r\n\r\nPUSH1 @ pc=2\r\nstack: [6, 7]\r\nmemory:[]\r\n\r\nMUL @ pc=4\r\nstack: [42]\r\nmemory:[]\r\n\r\nTraceback (most recent call last):\r\nsoul_evm.opcodes.InvalidCodeOffset: {'code': b'`\\x06`\\x07\\x02', 'pc': 5}\r\n\r\n```\r\n\r\n将会报`InvalidCodeOffset`的错误。\r\n\r\n---\r\n\r\n当我试图在字节指令的末尾处获取下一条指令时，遇到了一个错误。\r\n\r\n让我们来看看关于这个场景下，黄皮书说了什么？\r\n\r\n\r\n\r\n\r\n大致概括一下：如果程序计数器在代码字节数组之外，那么将会干净利落地停止，而不是抛出异常，所以我们需要改变解码器中的条件：\r\n\r\n`opcodes.py/decode_opcode`\r\n\r\n```python\r\ndef decode_opcode(context: ExecutionContext) -> Instruction:\r\n if context.pc < 0:\r\n raise InvalidCodeOffset({\"code\": context.code.hex(), \"pc\": context.pc})\r\n\r\n if context.pc >= len(context.code):\r\n return STOP\r\n\r\n #通过bytes获取指令\r\n opcode = context.read_code(1)\r\n instruction = INSTRUCTIONS_BY_OPCODE.get(opcode)\r\n if instruction is None:\r\n raise UnknownOpcode({\"opcode\": opcode})\r\n\r\n return instruction\r\n```\r\n\r\n如此一来，便不会报错了！\r\n\r\n```shell\r\nPS E:evmproject\\src> python main.py 6006600702\r\nPUSH1 @ pc=0\r\nstack: [6]\r\nmemory:[]\r\n\r\nPUSH1 @ pc=2\r\nstack: [6, 7]\r\nmemory:[]\r\n\r\nMUL @ pc=4\r\nstack: [42]\r\nmemory:[]\r\n\r\nSTOP @ pc=5\r\nstack: [42]\r\nmemory:[]\r\n\r\nOutput: 0x\r\n\r\n```\r\n\r\n\r\n\r\n## 🌽返回数据\r\n\r\n> 到目前为止，我们一直在监视跟踪运行中的堆栈，但在实际应用中，以太坊虚拟机必须返回值才算有用。\r\n>\r\n> 根据黄皮书，返回值意味着什么？\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n返回从堆栈中弹出的两个元素，`offset`和`length`以及返回的`memory[offset:offset+length-1]`\r\n\r\n\r\n\r\n---\r\n\r\n所以为了返回数据，我们需要实现内存存储指令和`RETURN`。让我们从`MSTORE8`开始，它能够从堆栈中弹出一个偏移量和一个字，并将该字的最低字节存储在内存中：\r\n\r\n\r\n\r\n\r\n```python\r\nMSTORE8 = register_instruction(\r\n 0x53,\r\n \"MSTORE8\",\r\n (lambda ctx: ctx.memory.store(ctx.stack.pop(), ctx.stack.pop() % 256)),\r\n)\r\n```\r\n\r\n为了`RETURN`，我们添加一个`load_range(offset,length)` 来操作临时存储：\r\n\r\n```python\r\ndef load_range(self, offset: int, length: int) -> bytes:\r\n if offset < 0:\r\n raise InvalidMemoryAccess({\"offset\": offset, \"length\": length})\r\n\r\n # we could use a slice here, but this lets us gets 0 bytes if we read past the end of concrete memory\r\n return bytes(self.load(x) for x in range(offset, offset + length))\r\n```\r\n\r\nnote：添加在`Memory`类中\r\n\r\n\r\n\r\n然后我们就可以给我们的`ExcutionContext`加上获取返回数据的属性函数：\r\n\r\n```python\r\nclass ExecutionContext:\r\n def __init__(self, code=bytes(), pc=0, stack=Stack(), memory=Memory()) -> None:\r\n ...\r\n self.returndata = bytes()\r\n\r\n def set_return_data(self, offset: int, length: int) -> None:\r\n self.stopped = True\r\n self.returndata = self.memory.load_range(offset, length)\r\n```\r\n\r\n\r\n\r\n现在给出`RETURN`的指令：\r\n\r\n```python\r\nRETURN = register_instruction(\r\n 0xf3,\r\n \"RETURN\",\r\n (lambda ctx: ctx.set_return_data(ctx.stack.pop(), ctx.stack.pop())),\r\n)\r\n```\r\n\r\n\r\n\r\n现在我们就可以返回并且输出运行之后的结果啦！\r\n\r\n```shell\r\nPS E:evmproject\\src> python main.py 600660070260005360016000f3\r\nPUSH1 @ pc=0\r\nstack: [6]\r\nmemory:[]\r\n\r\nPUSH1 @ pc=2\r\nstack: [6, 7]\r\nmemory:[]\r\n\r\nMUL @ pc=4\r\nstack: [42]\r\nmemory:[]\r\nPUSH1 @ pc=8\r\nstack: [1]\r\nmemory:[42]\r\n\r\nPUSH1 @ pc=10\r\nstack: [1, 0]\r\nmemory:[42]\r\n\r\nRETURN @ pc=12\r\nstack: []\r\nmemory:[42]\r\n\r\nOutput: 0x2a\r\n```\r\n\r\n\r\n\r\n> 翻译不易，码字不简~\r\n>\r\n> 有条件的同学可以点赞收藏加关注~这是对博主最大的肯定！\r\n\r\n## 💎个人博客(合约安全审计方向)\r\n\r\n[77Brother (nangbowan.github.io)](https://nangbowan.github.io/)\r\n\r\n## 原文链接\r\n\r\n[Building an EVM from scratch part 1 - the execution context (notion.so)](https://www.notion.so/Building-an-EVM-from-scratch-part-1-the-execution-context-c28ebb4200c94f6fb75948a5feffc686)\r\n\r\n[以太坊黄皮书-yellow paper](https://ethereum.github.io/yellowpaper/paper.pdf)"},"author":{"user":"https://learnblockchain.cn/people/10353","address":null},"history":"QmdRZH4uj6N2aWDJuequiTZuwxS2EHVWJM6RyhjiZNpL1P","timestamp":1667813022,"version":1}