{"content":{"title":"Web3 Devops with Azure Devops pipeline 2","body":"# Web3 Devops with Azure Devops pipeline 2\r\n\r\n![VSTS_workflow.png](https:\/\/img.learnblockchain.cn\/attachments\/2022\/09\/gr9jdTAG63281474bc380.png)\r\n\r\n在[上一篇文章](https:\/\/learnblockchain.cn\/article\/4652)中，设计了pipelines的6个阶段，并完成了build部分的实现：\r\n\r\n1、build：编译、测试和打包工件\r\n\r\n2、dev：部署基础设施、合约和前端\r\n\r\n3、dev_validation：等待手动验证dev并删除dev环境\r\n\r\n4、qa：部署基础设施、合约和前端\r\n\r\n5、qa_validation 等待手动验证 qa 并删除 qa 环境\r\n\r\n6、prod：部署基础设施、合约和前端\r\n\r\n---\r\n\r\n整体看下来大概是这样一个流程：\r\n\r\n\r\n![azure_devops2.drawio.png](https:\/\/img.learnblockchain.cn\/attachments\/2022\/09\/EV8rKm9v632812321e876.png)\r\n\r\n\r\n- ***部署的时候遇到个问题，所以在项目中又加入了一个API，详细问题后面再讲。***\r\n\r\n---\r\n\r\n### ***IaC (Infrastructure as Code)***\r\n\r\n因为测试项目的前端Dapp比较简单，所以使用Azure Static Web App来部署。SWA的部署是通过在Pipeline中使用脚本和模板来部署的，这种方式也称为基础设施即代码（IaC）（使用DevOps方法和版本控制与描述性模型来定义和部署基础设施，如网络、虚拟机、负载平衡器等等。就像相同的源代码总是生成相同的二进制文件一样，IaC模型每次部署时都会生成相同的环境）\r\n\r\nIaC的模板是使用Azure Bicep编写的。它是一种特定领域的语言（DSL），使用声明式语法来部署Azure资源。在Bicep文件中，可以定义要部署到Azure的基础设施，然后在整个开发生命周期中使用该文件以一种一致的方式来重复部署基础设施。\r\n\r\n在工程目录中新建iac文件夹，并创建 `saw.becip` 文件，在其中定义Azure SWA资源：\r\n\r\n```yaml\r\nparam repoUrl string\r\nparam location string\r\n\r\nresource swa 'Microsoft.Web\/staticSites@2022-03-01' = {\r\n name: 'swa${uniqueString(resourceGroup().id)}'\r\n location: location\r\n tags: {\r\n tagName1: 'demo'\r\n }\r\n sku: {\r\n name: 'Free'\r\n tier: 'Free'\r\n }\r\n properties: {\r\n branch: 'v2.1'\r\n repositoryToken: ''\r\n repositoryUrl: repoUrl\r\n buildProperties: {\r\n apiLocation: ''\r\n appLocation: '\/'\r\n appArtifactLocation: 'dist'\r\n }\r\n }\r\n}\r\n\r\noutput swaName string = swa.name\r\noutput deploymentToken string = swa.listSecrets().properties.apiKey\r\n```\r\n\r\n还需要在创建一个 `main.bechip` 来定义资源组：\r\n\r\n```yaml\r\ntargetScope = 'subscription'\r\n\r\nparam repoUrl string\r\nparam swaName string = 'web3swa'\r\nparam location string = 'eastasia'\r\nparam rgName string = 'truffle_demo'\r\n\r\nresource rg 'Microsoft.Resources\/resourceGroups@2021-04-01' = {\r\n name: rgName\r\n location: location\r\n}\r\n\r\nmodule web3swa '.\/saw.bicep' = {\r\n name: swaName\r\n scope: resourceGroup(rg.name)\r\n params: {\r\n repoUrl: repoUrl\r\n location: location\r\n }\r\n}\r\n\r\noutput swaName string = web3swa.outputs.swaName\r\noutput deploymentToken string = web3swa.outputs.deploymentToken\r\n```\r\n\r\n最后使用powershell来编写部署脚本 `deploy.ps1`：\r\n\r\n```powershell\r\n[CmdletBinding()]\r\nparam(\r\n # Parameter help description\r\n [Parameter(Position=0)]\r\n [string]\r\n $rgName=\"web3devops_dev\",\r\n\r\n # Parameter help description\r\n [Parameter(Position=1)]\r\n [string]\r\n $location=\"eastasia\",\r\n\r\n [string]\r\n $repoUrl\r\n)\r\n\r\nWrite-Verbose $rgName\r\nWrite-Output 'Deploying the Azure infrastructure'\r\n\r\n$deployment = $(az deployment sub create --name $rgName `\r\n --location $location `\r\n --template-file .\/main.bicep `\r\n --parameters rgName=$rgName `\r\n --parameters location=$location `\r\n --parameters repoUrl=$repoUrl `\r\n --output json) | ConvertFrom-Json\r\n\r\n$swaName = $deployment.properties.outputs.swaName.value\r\n$deploymentToken = $deployment.properties.outputs.deploymentToken.value\r\nWrite-Host \"##vso[task.setvariable variable=swaName;isOutput=true]$swaName\"\r\nWrite-Host \"##vso[task.setvariable variable=resourceGroup;isOutput=true]$rgName\"\r\nWrite-Host \"##vso[task.setvariable variable=deploymentToken;isOutput=true]$deploymentToken\"\r\n```\r\n\r\n修改之前的Azure pipelines文件，在Build阶段添加两个task，用于将iac文件打包并发布到Artifact：\r\n\r\n```yaml\r\n- task: CopyFiles@2\r\n displayName: Package IaC\r\n inputs:\r\n Contents: $(System.DefaultWorkingDirectory)\/myapp\/iac\/**\r\n TargetFolder: $(Build.ArtifactStagingDirectory)\/iac\r\n flattenFolders: true\r\n- task: PublishPipelineArtifact@1\r\n displayName: Publish IaC\r\n inputs:\r\n targetPath: \"$(Build.ArtifactStagingDirectory)\/iac\"\r\n artifact: \"iac\"\r\n publishLocation: \"pipeline\"\r\n```\r\n\r\n修改dev阶段的iac job，将部署脚本添加到Azure pipelines文件中：\r\n\r\n```yaml\r\n- job: iac\r\n steps:\r\n - checkout: none\r\n - download: none\r\n - task: DownloadPipelineArtifact@2\r\n displayName: \"Download IaC artifacts\"\r\n inputs:\r\n buildType: \"current\"\r\n artifact: \"iac\"\r\n targetPath: \"$(Pipeline.Workspace)\/iac\"\r\n - task: AzureCLI@2\r\n name: \"deploy\"\r\n displayName: \"Deploy Infra\"\r\n inputs:\r\n azureSubscription: \"web3devops\"\r\n scriptType: \"pscore\"\r\n scriptLocation: \"scriptPath\"\r\n scriptPath: \"$(Agent.BuildDirectory)\/iac\/deploy.ps1\"\r\n arguments: \"-repoUrl $(Build.Repository.Uri) -rgName $(resourceGroup)-dev -verbose\"\r\n workingDirectory: \"$(Agent.BuildDirectory)\/iac\"\r\n```\r\n\r\n这样每次部署业务代码之前，都会先部署基础架构服务\r\n\r\n---\r\n\r\n### API (Azure Function)\r\n\r\n这里就要讲到一开始提到的那个问题了，也是加入这个API的原因。\r\n\r\n在测试部署的时候遇到了一个问题，合约编译后的ABI文件(Application Binary Interface，JSON 文件。描述了已部署的合约及其功能)需要在Dapp中引用以便获取Contract Address，但在合约部署之前ABI文件中没有Contract Address信息，这部分内容是在合约部署之后自动添加到ABI文件中的。\r\n\r\n在本地测试的时候，使用软链接的方式把合约的编译目录链到了Dapp的src目录，每次发布合约后再编译Dapp进行前端页面的测试，所以不会遇到这个问题\r\n\r\n```bash\r\nln -s \/mycontract\/build\/contracts \/myapp\/src\/contracts\r\n```\r\n\r\n但是在pipelines里，是需要在Build阶段编译Contract和Dapp，然后发布到Actifact。这样就没有办法等到合约部署完成之后再编译Dapp。所以想到了另外一种方法来解决，流程图如下：\r\n\r\n\r\n![Azure_Devops.drawio.png](https:\/\/img.learnblockchain.cn\/attachments\/2022\/09\/Tc6xd7Gx6328129816f43.png)\r\n\r\n\r\n1、部署合约\r\n\r\n2、合约部署成功后会将Contract Address写回到ABI文件\r\n\r\n3、在Pipelines中解析ABI文件，拿到Network ID和Contract Address，并添加到Pipelines变量中\r\n\r\n4、部署Dapp到SWA\r\n\r\n5、部署API到Azure Function，并将Pipelines变量中的值设置成Function的环境变量\r\n\r\n6、当浏览器打开Dapp的时候，会调用GET方法调用API获取Network ID和Contract Address\r\n\r\n7、从metamask钱包中读取网络地址和用户地址\r\n\r\n8、执行交易并查看结果\r\n\r\n首先要在vscode中安装一个插件\r\n\r\n![Untitled.png](https:\/\/img.learnblockchain.cn\/attachments\/2022\/09\/RSQxhKoH632812ba5268c.png)\r\n\r\n使用 `CTRL + SHIFT + P`快捷键，打开vscode快捷命令行，执行：\r\n\r\n```yaml\r\nAzure Static Web App: Create HTTP Function\r\n```\r\n\r\n根据提示创建一个JavaScript Function工程，并切换到工程目录修改 `index.js` 文件。这是一个非常简单的服务，返回一个从环境变量中读取的值：\r\n\r\n```jsx\r\nmodule.exports = async function (context, req) {\r\n context.log('GetContractAddress: process a request.');\r\n const networkId = (req.query.networkId || undefined);\r\n\r\n if(!networkId) {\r\n context.res = {\r\n status: 400\r\n };\r\n return;\r\n }\r\n\r\n const address = process.env[`networkAddress_${networkId}`]\r\n\r\n context.log(`Returning: ${address}`)\r\n\r\n context.res = {\r\n \/\/ status: 200, \/* Defaults to 200 *\/\r\n body: address\r\n };\r\n}\r\n```\r\n\r\nAPI使用 dotenv 来管理环境变量，并使用 Jest 作为测试框架，所以编写测试方法需要安装 `jest` 和 `jest-junit`\r\n\r\n```bash\r\nnpm install dotenv jest jest-junit\r\n```\r\n\r\n传递给Function的第一个参数，是一个暴露了日志函数的上下文对象。为了模拟上下文对象，在API文件夹下创建了TestMocks文件夹，并在此文件夹中创建了一个名为 `defaultContext.js` 的文件，用于模拟的上下文对象，来跟踪调用以便记录和检查消息\r\n\r\n```jsx\r\nmodule.exports = {\r\n log: jest.fn()\r\n};\r\n```\r\n\r\n现在可以编写测试了，在工程目录新建测试文件 `index.test.js` ，并添加以下内容：\r\n\r\n```jsx\r\nrequire('dotenv').config();\r\nconst httpFunction = require('.\/index');\r\nconst context = require('..\/TestMocks\/defaultContext');\r\n\r\nafterEach(()=>{\r\n jest.clearAllMocks();\r\n});\r\n\r\ntest('Should return address', async () => {\r\n const request = {\r\n query: { networkId: 4 }\r\n };\r\n\r\n await httpFunction(context, request);\r\n\r\n expect(context.log.mock.calls.length).toBe(2);\r\n expect(context.res.body).toEqual('0x7a063c7e4A0EC2fB4dC0F73103Fd45F17b46Ae52');\r\n});\r\n\r\ntest('Should return 400 error',async () => {\r\n const request = {\r\n query: {}\r\n };\r\n\r\n await httpFunction(context, request);\r\n\r\n expect(context.log.mock.calls.length).toBe(1);\r\n expect(context.res.status).toEqual(400);\r\n});\r\n```\r\n\r\n另外在本地测试为了模拟环境变量，还需要增加.env文件：\r\n\r\n```jsx\r\nnetworkAddress_4=0x7a063c7e4A0EC2fB4dC0F73103Fd45F17b46Ae52\r\n```\r\n\r\n执行测试\r\n\r\n![Untitled 1.png](https:\/\/img.learnblockchain.cn\/attachments\/2022\/09\/0x3999K1632812db66fed.png)\r\n\r\n修改Azure Pipelines文件\r\n\r\n1、在Build阶段 `Publish frontend` 之后增加以下内容，用于测试和发布API\r\n\r\n```yaml\r\n- task: PublishPipelineArtifact@1\r\n displayName: Publish API\r\n inputs:\r\n targetPath: \"$(System.DefaultWorkingDirectory)\/myapp\/api\"\r\n artifact: \"api\"\r\n publishLocation: \"pipeline\"\r\n- script: npm install\r\n displayName: \"Install API dependencies\"\r\n workingDirectory: $(System.DefaultWorkingDirectory)\/myapp\/api\r\n- script: npm test -- --reporters=default --reporters=jest-junit\r\n displayName: \"Test API\"\r\n workingDirectory: $(System.DefaultWorkingDirectory)\/myapp\/api\r\n env:\r\n CI: true\r\n networkAddress_4: \"0x7a063c7e4A0EC2fB4dC0F73103Fd45F17b46Ae52\"\r\n- task: PublishTestResults@2\r\n displayName: \"Publish API test results\"\r\n inputs:\r\n testRunTitle: \"API\"\r\n testResultsFormat: \"JUnit\"\r\n failTaskOnFailedTests: true\r\n testResultsFiles: \"myapp\/api\/junit*.xml\"\r\n```\r\n\r\n2、在dev阶段，需要修改 `deploy_contract` 和 `deploy_frontend` \r\n\r\n在contract部署之后需要解析ABI文件，并创建pipelines变量\r\n\r\n```yaml\r\n- job: deploy_contracts\r\n dependsOn: iac\r\n steps:\r\n - checkout: none\r\n - download: none\r\n - task: DownloadPipelineArtifact@2\r\n displayName: \"Download artifacts\"\r\n - script: npm install --global web3 ethereumjs-testrpc ganache-cli truffle\r\n displayName: \"Install Global npm package\"\r\n - script: npm install\r\n displayName: \"Install npm package\"\r\n workingDirectory: \"$(Agent.BuildDirectory)\/contracts\/contracts3\"\r\n - script: \"truffle migrate --reset --compile-none --network development\"\r\n displayName: \"Deploy contracts\"\r\n workingDirectory: \"$(Agent.BuildDirectory)\/contracts\/contracts3\"\r\n - pwsh: |\r\n $contract = Get-Content .\/build\/contracts\/TestNFT.json | ConvertFrom-Json\r\n\r\n $networkId = $contract.networks[0] | Get-Member -MemberType NoteProperty | Select-Object -ExpandProperty name\r\n\r\n $address = $contract.networks.$networkId.address\r\n\r\n if($address) {\r\n Write-Host \"##[section]Contract address: $address\"\r\n } else {\r\n Write-Host \"##vso[task.logissue type=error]Address not found\"\r\n }\r\n\r\n Write-Host \"##vso[task.setvariable variable=networkId;isOutput=true]$networkId\"\r\n Write-Host \"##vso[task.setvariable variable=contractAddress;isOutput=true]$address\"\r\n displayName: \"Find contract address\"\r\n name: \"Contract\"\r\n workingDirectory: \"$(Agent.BuildDirectory)\/contracts\/contracts3\"\r\n```\r\n\r\n在frontend部署的时候会读取变量，并设置API的环境变量\r\n\r\n```yaml\r\n- job: deploy_frontend\r\n dependsOn: \r\n - iac\r\n - deploy_contracts\r\n variables:\r\n swaName: $[ dependencies.IaC.outputs['deploy.swaName'] ]\r\n deploymentToken: $[ dependencies.IaC.outputs['deploy.deploymentToken'] ]\r\n networkId: $[ dependencies.Deploy_Contracts.outputs['contract.networkId'] ]\r\n contractAddress: $[ dependencies.Deploy_Contracts.outputs['contract.contractAddress'] ]\r\n steps:\r\n - checkout: none\r\n - download: none\r\n - task: DownloadPipelineArtifact@2\r\n displayName: Download artifacts\r\n - task: AzureStaticWebApp@0\r\n displayName: Deploy frontend\r\n inputs:\r\n api_location: api\r\n app_location: client\r\n skip_app_build: true\r\n workingDirectory: $(Pipeline.Workspace)\r\n azure_static_web_apps_api_token: $(deploymentToken)\r\n - task: AzureCLI@2\r\n displayName: \"Configure API\"\r\n inputs:\r\n azureSubscription: \"Web3DevOps\"\r\n scriptType: \"pscore\"\r\n scriptLocation: \"inlineScript\"\r\n inlineScript: |\r\n az staticwebapp appsettings set --name $(swaName) `\r\n --setting-names networkAddress=$(contractAddress) `\r\n networkAddress_$(networkId)=$(contractAddress)\r\n - task: AzureCLI@2\r\n displayName: \"Update summary\"\r\n inputs:\r\n azureSubscription: \"Web3DevOps\"\r\n scriptType: \"pscore\"\r\n scriptLocation: \"inlineScript\"\r\n inlineScript: |\r\n dir env: | Out-String\r\n\r\n Write-Host \"resourceGroup: $(resourceGroup)\"\r\n\r\n $summaryPath = \"$(Agent.BuildDirectory)\/contracts\/Contract_Information.md\"\r\n $swaUrl = $env:AZURESTATICWEBAPP_STATIC_WEB_APP_URL\r\n Write-Host \"Writing summary to $summaryPath\"\r\n\r\n $data = @\"\r\n ### SPA Information\r\n App URL: [$swaUrl]($swaUrl)\r\n \"@\r\n\r\n Set-Content -Path $summaryPath -Value $data -Verbose\r\n $cmd = '[task.addattachment type=Distributedtask.Core.Summary;name=dApp Information;]'\r\n\r\n Write-Host \"##vso$cmd$summaryPath\"\r\n```\r\n\r\n这样就可以在浏览器打开dapp的时候，从API中获取合约地址\r\n\r\n---\r\n\r\n### DApps(Decentralized Applications)\r\n\r\n在上一篇文章中，我把两个测试帐号以及测试网络地址都Hardcoded在了页面里，这次修改为从钱包中读取网络地址和用户帐号，只保留了一个交易对方的账户\r\n\r\n```jsx\r\n\r\nimport '.\/App.css';\r\nimport React, {Component} from \"react\";\r\nimport testnft from \".\/contracts\/TestNFT.json\";\r\n\r\nclass App extends Component{\r\n state = { storageValue: null, web3: null, account: null, toaccount: null, contract: null };\r\n\r\n componentDidMount = async () =>{\r\n try{\r\n \/\/Get network provider and web3 instance.\r\n const Web3 = require('web3');\r\n const toaccount = '0x212b13B538e25F16A80D2e9e1B9CC1c1aeCBe279';\r\n const web3 = new Web3(window.ethereum);\r\n\r\n \/\/Use web3 to get the user's accounts.\r\n const accounts = await window.ethereum.request({ method: 'eth_requestAccounts' });\r\n \r\n \/\/Get the contract instance.\r\n const networkId = await web3.eth.net.getId(); \r\n const deployedNetwork = testnft.networks[networkId]; \r\n let contractAddress = deployedNetwork && deployedNetwork.address; \r\n \r\n \/\/ If the network can't be found in the contract JSON call the\r\n \/\/ backend API for the address.\r\n if(!contractAddress){\r\n console.log('Address not found in contract JSON. Calling backup api');\r\n const text = await (await fetch(`\/api\/GetContractAddress\/?networkId=${networkId}`)).text(); \r\n contractAddress = text;\r\n }\r\n\r\n const instance = new web3.eth.Contract(\r\n testnft.abi,\r\n contractAddress,\r\n );\r\n\r\n \/\/ Set web3, accounts, and contract to the state, and then proceed with an\r\n \/\/ example of interacting with the contract's methods.\r\n this.setState({ web3, accounts, toaccount, contract: instance }, this.runExample);\r\n }catch(error){\r\n alert(\r\n `Failed to load web3, accounts, or contract. Check console for details.`,\r\n );\r\n console.error(error);\r\n }\r\n };\r\n\r\n runExample = async () => {\r\n const { web3, accounts, toaccount, contract } = this.state;\r\n console.log(accounts[0]);\r\n console.log(toaccount);\r\n \r\n const response = await contract.methods.mintNFT('ipfs:\/\/QmYGkmHAhySYR6zvizG3xoMEyLPs2h68swsg5BeStxL5uK', toaccount).send( {from: accounts[0], gasLimit: 3500000, gasPrice: web3.utils.toWei('3', 'Gwei')} );\r\n this.setState({ storageValue: 'from: '+ response['from'] + '. to: ' + response['to'] + '. In the block: ' + response['blockNumber'] });\r\n };\r\n\r\n render() {\r\n if (!this.state.web3) {\r\n return
!Loading Web3, accounts, and contract...{this.state.web3},test<\/div>;\r\n }\r\n return (\r\n
\r\n

Good to Go!<\/h1>\r\n

Your Truffle Box is installed and ready.<\/p>\r\n

Smart Contract Example<\/h2>\r\n

\r\n This is a simple demo of casting NFT, if successfully executed, \r\n will print the from<\/strong> and to<\/strong> addresses, \r\n and the blocknumber<\/strong>\r\n <\/p>\r\n

\r\n Try changing the value stored on runExample()<\/strong> of App.js.\r\n <\/p>\r\n

The stored value is: {this.state.storageValue}<\/div>\r\n <\/div>\r\n );\r\n }\r\n}\r\n\r\nexport default App;\r\n```\r\n\r\n---\r\n\r\n### Azure Pipelines\r\n\r\n解决了dev阶段的部署问题之后，在dev_validation阶段，还要以来人工手动来验证。在pipelines中设置了等待时间，在超过等待时间后，会清除环境\r\n\r\n```yaml\r\n- stage: dev_validation\r\n dependsOn: dev\r\n jobs:\r\n - job: wait_for_dev_validation\r\n displayName: \"Wait for external validation\"\r\n pool: server\r\n timeoutInMinutes: 1440 # job times out in 1 day\r\n steps:\r\n - task: ManualValidation@0\r\n timeoutInMinutes: 1440 # task times out in 1 day\r\n inputs:\r\n notifyUsers: $(Build.RequestedForEmail)\r\n instructions: Use the App URL on the Extensions tab and validate the recent changes to your dApp and click resume.\r\n onTimeout: reject\r\n - job: delete_dev\r\n dependsOn: wait_for_dev_validation\r\n steps:\r\n - task: AzureCLI@2\r\n displayName: \"Delete Dev resource group\"\r\n inputs:\r\n azureSubscription: \"Web3DevOps\"\r\n scriptType: \"pscore\"\r\n scriptLocation: \"inlineScript\"\r\n inlineScript: \"az group delete --name $(resourceGroup)-dev --yes --no-wait\"\r\n```\r\n\r\nQA和Prod的流程基本一致，只是合约部署的位置不同。修改 `truffle-config.js` 文件，添加 `Dev、QA、Prod` 环境的定义，部署时选择各自的环境。\r\n\r\n```jsx\r\nnetworks: {\r\n \/\/ Useful for testing. The `development` name is special - truffle uses it by default\r\n \/\/ if it's defined here and no other network is specified at the command line.\r\n \/\/ You should run a client (like ganache, geth, or parity) in a separate terminal\r\n \/\/ tab if you use this network and you must also set the `host`, `port` and `network_id`\r\n \/\/ options below to some value.\r\n \/\/\r\n \r\n dev_development: {\r\n provider: () => new HDWalletProvider(mnemonic, `https:\/\/chain.azure-api.net\/testrpc`), \r\n network_id: \"5\", \/\/ Any network (default: none)\r\n },\r\n\r\n\t\tqa_development: {\r\n provider: () => new HDWalletProvider(mnemonic, `https:\/\/goerli.infura.io\/v3\/******`),\r\n network_id: \"1337\", \/\/ Any network (default: none)\r\n },\r\n\r\n\t\t\r\n\t\tpord_development: {\r\n provider: () => new HDWalletProvider(mnemonic, `https:\/\/goerli.infura.io\/v3\/******`),\r\n network_id: \"1\", \/\/ Any network (default: none)\r\n },\r\n },\r\n```\r\n\r\n- ***最后关于Quorum Dev Quickstart的部署，可以看看我之前写的部署方案。***"},"author":{"user":"https:\/\/learnblockchain.cn\/people\/9998","address":null},"history":"QmVF3GLDggCq6BvRKqpChxkmXtJno43QnQJ2y3LkPi8XKr","timestamp":1663665074,"version":1}