New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Pick a username
Email Address
Password
Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[typescript] Convert ABIs to TypeScript #14

Closed
rojosnow opened this Issue May 29, 2018 · 21 comments

Comments

Member

rojosnow commented May 29, 2018
edited

User Story

As a developer, I need access to the ABIs using TypeScript so that my code will meet the project's standards.

Why Is this Needed?

Summary: The current json ABIs need to be converted to TypeScript.

Description

Type: Feature

Current Behavior

No TypeScript ABIs.

Expected Behavior

ABIs available in TypeScript.

Solution

Summary: Using the market-solidity npm package, convert the json ABIs to TypeScript. The package is installed as a dev dependency. The json contracts are located at node_modules/market-solidity/build/contracts.

Try using 0x's ABI generator, https://github.com/0xProject/0x-monorepo/tree/development/packages/abi-gen. Additional details are listed here, https://blog.0xproject.com/abi-to-typescript-generator-b0fb5cae9e29.

If possible, use the standard templates, https://github.com/0xProject/0x-monorepo/tree/v2-prototype/packages/contract_templates. If not, modify/create templates for the ABIs.

Use the types repo and store the templates in the templates directory and type definitions in the types directory at the root level.

Definition of Done

  • Convert json ABIs in the node_modules/market-solidity/build/contracts to their TypeScript equivalents.
  • Store the resulting TypeScript ABIs in thetypes directory in the types repo.
  • Create or modify the templates as needed.
  • Store the resulting templates in the templates directory in the types repo...even if the templates are unchanged from the 0x project.
  • Test and verify the type definitions.
  • Modify package.json to generate type definitions.

@rojosnow rojosnow added this to Backlog in MARKET Protocol via automation May 29, 2018

@pelsasser pelsasser referenced this issue May 29, 2018

Merged

Feature/dev environment #5 #15

3 of 3 tasks complete

rojosnow added a commit that referenced this issue May 29, 2018

@pelsasser pelsasser added this to the Alpha Release milestone May 30, 2018

@pelsasser pelsasser changed the title from Convert ABIs to TypeScript to Convert ABIs to TypeScript using 0x abi-gen May 30, 2018

This comment has been minimized.

Show comment
Hide comment
@gitcoinbot

gitcoinbot May 30, 2018

Issue Status: 1. Open 2. Started 3. Submitted 4. Done


This issue now has a funding of 0.25 ETH (141.28 USD @ $565.13/ETH) attached to it.

gitcoinbot commented May 30, 2018

Issue Status: 1. Open 2. Started 3. Submitted 4. Done


This issue now has a funding of 0.25 ETH (141.28 USD @ $565.13/ETH) attached to it.

This comment has been minimized.

Show comment
Hide comment
@gitcoinbot

gitcoinbot May 30, 2018

Issue Status: 1. Open 2. Started 3. Submitted 4. Done


Work has been started.

These users each claimed they can complete the work by 4 days, 12 hours ago. Please review their questions below:

  1. ConnorChristie has started work.
  • Q: I can begin working on these templates!

gitcoinbot commented May 30, 2018
edited

Issue Status: 1. Open 2. Started 3. Submitted 4. Done


Work has been started.

These users each claimed they can complete the work by 4 days, 12 hours ago. Please review their questions below:

  1. ConnorChristie has started work.
  • Q: I can begin working on these templates!

rojosnow pushed a commit that referenced this issue May 30, 2018

Robert Jordan
Update module project structure (#11)
* Structure API project into modules.

* Added additional modules, Util, SimEx, to the project.

* Added web3.

* Fixed failing test.

* Update package lock file, keep project on the 8.x.x LTS version of Node.

* Downgrade web3 to v0.20.6

* Add the supporting dev dependencies and files for the ABI json to ts issue #14.

* Feature/dev environment #5 (#15)

* add truffle config for dev env

* add makefile

* updated readme

* add section on migration
Verified

This comment has been minimized.

Show comment
Hide comment
@ConnorChristie

ConnorChristie May 30, 2018

@rojosnow So it looks like 0x's abi-gen requires a lot of their libraries to actually run and function:

Do you want these libraries to be added as dependencies or should I look for an alternative and cleaner way to generate the typings? Or are you strictly only wanting to use abi-gen?

import { BaseContract } from '@0xproject/base-contract';
import { ContractArtifact } from '@0xproject/sol-compiler';
import { BlockParam, BlockParamLiteral, CallData, ContractAbi, DataItem, MethodAbi, Provider, TxData, TxDataPayable } from '@0xproject/types';
import { BigNumber, classUtils, logUtils, promisify } from '@0xproject/utils';
import { Web3Wrapper } from '@0xproject/web3-wrapper';
import * as ethers from 'ethers';
import * as _ from 'lodash';

ConnorChristie commented May 30, 2018

@rojosnow So it looks like 0x's abi-gen requires a lot of their libraries to actually run and function:

Do you want these libraries to be added as dependencies or should I look for an alternative and cleaner way to generate the typings? Or are you strictly only wanting to use abi-gen?

import { BaseContract } from '@0xproject/base-contract';
import { ContractArtifact } from '@0xproject/sol-compiler';
import { BlockParam, BlockParamLiteral, CallData, ContractAbi, DataItem, MethodAbi, Provider, TxData, TxDataPayable } from '@0xproject/types';
import { BigNumber, classUtils, logUtils, promisify } from '@0xproject/utils';
import { Web3Wrapper } from '@0xproject/web3-wrapper';
import * as ethers from 'ethers';
import * as _ from 'lodash';

This comment has been minimized.

Show comment
Hide comment
@eswarasai

eswarasai May 30, 2018

Member

@ConnorChristie -- We're not necessarily stuck with abi-gen. It'd be great if we can find an easy alternative to quickly generate the typings for the ABIs. Let us know your approach here before you proceed with it. Thanks!

Member

eswarasai commented May 30, 2018
edited

@ConnorChristie -- We're not necessarily stuck with abi-gen. It'd be great if we can find an easy alternative to quickly generate the typings for the ABIs. Let us know your approach here before you proceed with it. Thanks!

This comment has been minimized.

Show comment
Hide comment
@rojosnow

rojosnow May 30, 2018

Member

@ConnorChristie Yes, as @eswarasai said, it's ok to use another solution. I'd like to get this process as automated as possible. I realize that might mean you're going to end up with a number of devDependencies to produce the output.

Member

rojosnow commented May 30, 2018

@ConnorChristie Yes, as @eswarasai said, it's ok to use another solution. I'd like to get this process as automated as possible. I realize that might mean you're going to end up with a number of devDependencies to produce the output.

@rojosnow rojosnow moved this from Backlog to In progress in MARKET Protocol May 30, 2018

This comment has been minimized.

Show comment
Hide comment
@ConnorChristie

ConnorChristie May 31, 2018

@eswarasai @rojosnow I have come across this package that only relies on web3 and itself while also generating a very streamline type file for each contract compared to abi-gen.

https://github.com/Neufund/TypeChain

Let me know what you think of that package! In addition, what tests would you like to see as part of this PR? Some unit tests possibly using truffle to deploy and exercise the contract methods?

ConnorChristie commented May 31, 2018

@eswarasai @rojosnow I have come across this package that only relies on web3 and itself while also generating a very streamline type file for each contract compared to abi-gen.

https://github.com/Neufund/TypeChain

Let me know what you think of that package! In addition, what tests would you like to see as part of this PR? Some unit tests possibly using truffle to deploy and exercise the contract methods?

This comment has been minimized.

Show comment
Hide comment
@eswarasai

eswarasai May 31, 2018

Member

@ConnorChristie -- I think this is the perfect solution for us. I'm not sure if we need any tests here. I'd revert to @rojosnow or @pelsasser on that matter. Thanks!

Member

eswarasai commented May 31, 2018

@ConnorChristie -- I think this is the perfect solution for us. I'm not sure if we need any tests here. I'd revert to @rojosnow or @pelsasser on that matter. Thanks!

This comment has been minimized.

Show comment
Hide comment
@rojosnow

rojosnow May 31, 2018

Member

@ConnorChristie The TypeChain tool looks like a good start. Once the typedefs are generated from the ABIs, we need to verify they actually work. @eswarasai Thoughts on the easiest way to do that? It might require we pull in the local dev environment work that @pelsasser did on feature/dev-environment#5

Member

rojosnow commented May 31, 2018

@ConnorChristie The TypeChain tool looks like a good start. Once the typedefs are generated from the ABIs, we need to verify they actually work. @eswarasai Thoughts on the easiest way to do that? It might require we pull in the local dev environment work that @pelsasser did on feature/dev-environment#5

This comment has been minimized.

Show comment
Hide comment
@eswarasai

eswarasai May 31, 2018

Member

@rojosnow -- For test cases, we can verify by confirming if the generated ts file has the set of type definitions + functions that are be expected to be used within the project.

@ConnorChristie -- How would you suggest we can accomplish that?

@pelsasser -- I'm wondering maybe we should move all this either to MARKETProtocol repo and add types there itself or create a new repo where we have all the typings of MARKETProtocol defined, instead of keeping them in MARKET.js.

Member

eswarasai commented May 31, 2018

@rojosnow -- For test cases, we can verify by confirming if the generated ts file has the set of type definitions + functions that are be expected to be used within the project.

@ConnorChristie -- How would you suggest we can accomplish that?

@pelsasser -- I'm wondering maybe we should move all this either to MARKETProtocol repo and add types there itself or create a new repo where we have all the typings of MARKETProtocol defined, instead of keeping them in MARKET.js.

This comment has been minimized.

Show comment
Hide comment
@rojosnow

rojosnow May 31, 2018

Member

@eswarasai Got you. I think separating this is an excellent idea. There is having the expected functions but you don't know if there are possible errors until you actually call them. Maybe is should live with MARKETProtocol so it can run a complete set of tests including using the typedefs.

Member

rojosnow commented May 31, 2018

@eswarasai Got you. I think separating this is an excellent idea. There is having the expected functions but you don't know if there are possible errors until you actually call them. Maybe is should live with MARKETProtocol so it can run a complete set of tests including using the typedefs.

This comment has been minimized.

Show comment
Hide comment
@pelsasser

pelsasser May 31, 2018

Member

my preference would be to have a seperated repo I think, not convinced of this, but it would allow us to separate the testing and use a much scaled down test environment from the MARKETProtocol behemoth

Member

pelsasser commented May 31, 2018

my preference would be to have a seperated repo I think, not convinced of this, but it would allow us to separate the testing and use a much scaled down test environment from the MARKETProtocol behemoth

This comment has been minimized.

Show comment
Hide comment
@eswarasai

eswarasai May 31, 2018

Member

@pelsasser -- I've just created a repo types for this task.

@ConnorChristie -- Could you please fork the repo types and setup everything there? Also please do let us know how you'd like to proceed in order to get tests running. Thanks!

Member

eswarasai commented May 31, 2018

@pelsasser -- I've just created a repo types for this task.

@ConnorChristie -- Could you please fork the repo types and setup everything there? Also please do let us know how you'd like to proceed in order to get tests running. Thanks!

This comment has been minimized.

Show comment
Hide comment
@ConnorChristie

ConnorChristie May 31, 2018

@eswarasai I think the idea of having the separate types repo is a very good idea! As for the testing, it might be a bit tricky because we can either:

  • run e2e tests through truffle by actually deploying contracts to a test instance which would ensure the types are generated from the actual solidity ABI, or
  • mock out the web3 instance passed to the types and use the unit tests to ensure everything matches the solidity contracts

Either way, the tests will require the knowledge of all the methods in each contract but the first actually tests the web3 interactions and the second only tests the interfaces.

Also, an upside to mocking out the web3 context is that we won't get caught up in dealing with the implementation of the contracts and will only need make sure the interfaces match and provide a valid passthrough. So if the contract implementation changes, these tests won't also have to be updated.

ConnorChristie commented May 31, 2018
edited

@eswarasai I think the idea of having the separate types repo is a very good idea! As for the testing, it might be a bit tricky because we can either:

  • run e2e tests through truffle by actually deploying contracts to a test instance which would ensure the types are generated from the actual solidity ABI, or
  • mock out the web3 instance passed to the types and use the unit tests to ensure everything matches the solidity contracts

Either way, the tests will require the knowledge of all the methods in each contract but the first actually tests the web3 interactions and the second only tests the interfaces.

Also, an upside to mocking out the web3 context is that we won't get caught up in dealing with the implementation of the contracts and will only need make sure the interfaces match and provide a valid passthrough. So if the contract implementation changes, these tests won't also have to be updated.

This comment has been minimized.

Show comment
Hide comment
@eswarasai

eswarasai Jun 1, 2018

Member

@ConnorChristie -- Thanks for putting out the possible options here. I'm in favour of having the web3 instance mocked out and just verifying whether the interfaces exist or not. Let us know if you need anything else in order to wrap this task. Thanks for being patient and helping us out here :)

Member

eswarasai commented Jun 1, 2018

@ConnorChristie -- Thanks for putting out the possible options here. I'm in favour of having the web3 instance mocked out and just verifying whether the interfaces exist or not. Let us know if you need anything else in order to wrap this task. Thanks for being patient and helping us out here :)

@rojosnow rojosnow changed the title from Convert ABIs to TypeScript using 0x abi-gen to [typescript] Convert ABIs to TypeScript using 0x abi-gen Jun 1, 2018

This comment has been minimized.

Show comment
Hide comment
@ConnorChristie

ConnorChristie Jun 2, 2018

@eswarasai Sounds good! The tests are going well but there is one minor thing, will you need types for every contract (i.e. Linkable, Ownable, ...) where they are parent contracts?

The reason I am asking is because the generated types include all the methods from their parents so the type classes have a lot of duplicated methods among them and it will take quite a while to write out all the tests for them. Maybe only include tests for the relevant contracts?

ConnorChristie commented Jun 2, 2018
edited

@eswarasai Sounds good! The tests are going well but there is one minor thing, will you need types for every contract (i.e. Linkable, Ownable, ...) where they are parent contracts?

The reason I am asking is because the generated types include all the methods from their parents so the type classes have a lot of duplicated methods among them and it will take quite a while to write out all the tests for them. Maybe only include tests for the relevant contracts?

This comment has been minimized.

Show comment
Hide comment
@eswarasai

eswarasai Jun 2, 2018

Member

@ConnorChristie -- Understood. For the time being, I think we can focus on having type-defs for the contracts that are already being used in dApp as mentioned here -- https://github.com/MARKETProtocol/dApp/blob/develop/src/Contracts.js

Member

eswarasai commented Jun 2, 2018

@ConnorChristie -- Understood. For the time being, I think we can focus on having type-defs for the contracts that are already being used in dApp as mentioned here -- https://github.com/MARKETProtocol/dApp/blob/develop/src/Contracts.js

@ConnorChristie ConnorChristie referenced this issue Jun 4, 2018

Merged

Convert ABIs to TypeScript #1

1 of 10 tasks complete

This comment has been minimized.

Show comment
Hide comment
@ConnorChristie

ConnorChristie Jun 4, 2018

@eswarasai PR has been opened but there are still a plethora of tests to add for all the other contracts, here is the link MARKETProtocol/types#1

I will continue working on the tests and update the PR as I go but if anyone else wants to help out, they may comment on the PR as to which tests they would like to help with!

ConnorChristie commented Jun 4, 2018

@eswarasai PR has been opened but there are still a plethora of tests to add for all the other contracts, here is the link MARKETProtocol/types#1

I will continue working on the tests and update the PR as I go but if anyone else wants to help out, they may comment on the PR as to which tests they would like to help with!

This comment has been minimized.

Show comment
Hide comment
@gitcoinbot

gitcoinbot Jun 6, 2018

Issue Status: 1. Open 2. Started 3. Submitted 4. Done


Work for 0.25 ETH (150.91 USD @ $603.65/ETH) has been submitted by:

  1. @ConnorChristie

@pelsasser please take a look at the submitted work:


gitcoinbot commented Jun 6, 2018

Issue Status: 1. Open 2. Started 3. Submitted 4. Done


Work for 0.25 ETH (150.91 USD @ $603.65/ETH) has been submitted by:

  1. @ConnorChristie

@pelsasser please take a look at the submitted work:


This comment has been minimized.

Show comment
Hide comment
@gitcoinbot

gitcoinbot Jun 6, 2018

Issue Status: 1. Open 2. Started 3. Submitted 4. Done


The funding of 0.25 ETH (151.63 USD @ $606.53/ETH) attached to this issue has been approved & issued to @ConnorChristie.

gitcoinbot commented Jun 6, 2018

Issue Status: 1. Open 2. Started 3. Submitted 4. Done


The funding of 0.25 ETH (151.63 USD @ $606.53/ETH) attached to this issue has been approved & issued to @ConnorChristie.

This comment has been minimized.

Show comment
Hide comment
@gitcoinbot

gitcoinbot Jun 6, 2018

⚡️ A tip worth 0.15 ETH (90.98 USD @ $606.53/ETH) has been granted to @ConnorChristie for this issue from pelsasser. ⚡️

The sender had the following public comments:

Thanks for helping out with getting all this set up, hope you will keep helping us out as we build out more of MARKET.js

Nice work @ConnorChristie! To redeem your tip, login to Gitcoin at https://gitcoin.co/explorer and select 'Claim Tip' from dropdown menu in the top right, or check your email for a link to the tip redemption page.

gitcoinbot commented Jun 6, 2018

⚡️ A tip worth 0.15 ETH (90.98 USD @ $606.53/ETH) has been granted to @ConnorChristie for this issue from pelsasser. ⚡️

The sender had the following public comments:

Thanks for helping out with getting all this set up, hope you will keep helping us out as we build out more of MARKET.js

Nice work @ConnorChristie! To redeem your tip, login to Gitcoin at https://gitcoin.co/explorer and select 'Claim Tip' from dropdown menu in the top right, or check your email for a link to the tip redemption page.

This comment has been minimized.

Show comment
Hide comment
@eswarasai

eswarasai Jun 6, 2018

Member

@ConnorChristie -- Thanks for all your hard work on getting the type-defs setup for the MARKET Contract ABIs. Would like to have many such contributions from you in the near future. Please do join our Discord channel :)

Member

eswarasai commented Jun 6, 2018

@ConnorChristie -- Thanks for all your hard work on getting the type-defs setup for the MARKET Contract ABIs. Would like to have many such contributions from you in the near future. Please do join our Discord channel :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment