Creating a Raw Transaction - Discussion

Hi Filip!

I was just curious, I am using linux. When I exectuted bitcoin-cli -regtest createrawtransaction with the transaction it wouldn’t create an error (error: Error parsing JSON: ), but when I removed the \ from the transaction and pasted it at the end of createrawtransaction it came back with a raw transaction. Am I missing something very obvious? thanks again :slight_smile:

1 Like

@filip was using Windows Powershell vs your Linux shell, that’s all, i think. Now if anyone can answer my query above…? https://forum.toshitimes.com/t/creating-a-raw-transaction-discussion/7910/36?u=billy

1 Like

So sorry for the lack of response here. It was some miscommunication between a team member and me.

Unfortunately you can’t use Bitcoin IDE for running Bitcoin transactons (oddly enough). There is a bug in one of Bitcoin IDE’s signature functions that makes this impossible. I only use it to teach about stack programming in general.

To decode a real bitcoin tx and go step by step through the code, you need a more advanced tool. I use this tool: https://github.com/kallewoof/btcdeb. But it’s quite complex and took my some time to figure it out myself.

2 Likes

Unfortunately different terminals and operation systems interpret the command line differently. Sometimes you need the \ to escape the quote characters and sometimes you do not.

Woah, that was intense… I see some people have already commented on this, but I do want to let you know that my Windows PowerShell doesn’t use the same commands. Is there anywhere we can go to find updated commands for PowerShell?

1 Like

Hi, I don’t use powershell, but have you tried to search google?
Please post the solution here when you find it, it may help other students having the same issue…

Thanx
Ivo

You can always install virtual box in Windows and experiment with some Linux distribution (inside a window of windows10)
You could install Ubuntu for example inside your windows. And just use this for learning bitcoin and other projects that works better on Linux. Or if you have on old laptop, you could wipe everything and install Ubuntu on it. On the mean time you learn more about Linux as well.
These things works much better on Linux, and if you choose Ubuntu, it’s not that hard to learn because it looks a bit like a windows version.

1 Like

one of the students has made a decent guide to install virtual box in windows to help others.

check it out here: https://forum.toshitimes.com/t/linux-unix-on-windows-system/12599/5?u=thecil

2 Likes

Hi @filip,

I’ve just gone through parts 3 and 4. As mentioned by others in previous posts, I also have a segwit transaction and so the signature is in the witness (“txinwitness”). Given this difference and following the steps that you outline in the video (for a non-Segwit transaction) is the following correct? You’ll see below that I’m then confused when it comes to the execution on the stack of the full script. Please can you clarify.

Starting with signing the transaction (that adds the Unlocking script to the raw transaction):

PS C:\Program Files\Bitcoin\daemon> ./bitcoin-cli -regtest signrawtransactionwithwallet 0200000001039661dc5e38b2f9410c67a1aed2df2e5d823a021fd2e3d9a705f9e3d6e0104e0000000000ffffffff01001110240100000017a9149169652648a37a8557cba3a7df7a139954f3dce88700000000
{
“hex”: “02000000000101039661dc5e38b2f9410c67a1aed2df2e5d823a021fd2e3d9a705f9e3d6e0104e00000000171600146dfd0a493ed681fe3e7efae5d05e6b7faba8219dffffffff01001110240100000017a9149169652648a37a8557cba3a7df7a139954f3dce8870247304402201ab25712385d15457b99a606fc22b8950a4b4f3bbc2f766a1fe1b44f560c1d490220621628cbf054bdf3b80453d163a5d13cda6980d28c95c20c0133bf15135d46ec01210350a7dc03c4353375ac0f8cbefcf6a64b1bbb2e2d27018f687487abe45c9cf58700000000”,
“complete”: true
}

Now copy the unlocking scripts ‘hex’ raw transaction and decoding it:
PS C:\Program Files\Bitcoin\daemon> ./bitcoin-cli -regtest decoderawtransaction 02000000000101039661dc5e38b2f9410c67a1aed2df2e5d823a021fd2e3d9a705f9e3d6e0104e00000000171600146dfd0a493ed681fe3e7efae5d05e6b7faba8219dffffffff01001110240100000017a9149169652648a37a8557cba3a7df7a139954f3dce8870247304402201ab25712385d15457b99a606fc22b8950a4b4f3bbc2f766a1fe1b44f560c1d490220621628cbf054bdf3b80453d163a5d13cda6980d28c95c20c0133bf15135d46ec01210350a7dc03c4353375ac0f8cbefcf6a64b1bbb2e2d27018f687487abe45c9cf58700000000
{
“txid”: “fe20c091ed354f18d87d1ee7728963b4de27c9f96764e2b7d38d50794ea356cb”,
“hash”: “5a9c0a17799b0ead5bee7314e27f8d2be4bed8c0f73fd671d6abed01de29beb8”,
“version”: 2,
“size”: 215,
“vsize”: 134,
“weight”: 533,
“locktime”: 0,
“vin”: [
{
“txid”: “4e10e0d6e3f905a7d9e3d21f023a825d2edfd2aea1670c41f9b2385edc619603”,
“vout”: 0,
“scriptSig”: {
“asm”: “00146dfd0a493ed681fe3e7efae5d05e6b7faba8219d”,
“hex”: “1600146dfd0a493ed681fe3e7efae5d05e6b7faba8219d”
},
“txinwitness”: [
“304402201ab25712385d15457b99a606fc22b8950a4b4f3bbc2f766a1fe1b44f560c1d490220621628cbf054bdf3b80453d163a5d13cda6980d28c95c20c0133bf15135d46ec01”,
“0350a7dc03c4353375ac0f8cbefcf6a64b1bbb2e2d27018f687487abe45c9cf587”
],
“sequence”: 4294967295
}
],
“vout”: [
{
“value”: 49.00000000,
“n”: 0,
“scriptPubKey”: {
“asm”: “OP_HASH160 9169652648a37a8557cba3a7df7a139954f3dce8 OP_EQUAL”,
“hex”: “a9149169652648a37a8557cba3a7df7a139954f3dce887”,
“reqSigs”: 1,
“type”: “scripthash”,
“addresses”: [
“2N6W6E5cbMvR8FYAzmJrhwAQheN6PeNGEMh”
]
}
}
]
}

Part 4: The unlocking script (i.e. first argument in the “txinwitness” field) is therefore: 304402201ab25712385d15457b99a606fc22b8950a4b4f3bbc2f766a1fe1b44f560c1d490220621628cbf054bdf3b80453d163a5d13cda6980d28c95c20c0133bf15135d46ec01

Now to obtain the locking script (given we have txid and vout from the decoded raw transaction above):

PS C:\Program Files\Bitcoin\daemon> ./bitcoin-cli -regtest gettxout “4e10e0d6e3f905a7d9e3d21f023a825d2edfd2aea1670c41f9b2385edc619603”, 0
{
“bestblock”: “0371a2f0c1a5a52e52d94da848f5473c77c184ee7c9c0cf54a5a34d82ac1e3bb”,
“confirmations”: 103,
“value”: 50.00000000,
“scriptPubKey”: {
“asm”: “OP_HASH160 bdbcbb6c06a27a9746739916348f0e0775d39f18 OP_EQUAL”,
“hex”: “a914bdbcbb6c06a27a9746739916348f0e0775d39f1887”,
“reqSigs”: 1,
“type”: “scripthash”,
“addresses”: [
“2NAYTnsqTPW3xnAuq9yVNmJW2UmK7Sya53P”
]
},
“coinbase”: true
}

The Unlocking Script ( given in the “asm” field) is therefore: OP_HASH160 bdbcbb6c06a27a9746739916348f0e0775d39f18 OP_EQUAL

The combined script ( Unlocking + Locking) is therefore:
304402201ab25712385d15457b99a606fc22b8950a4b4f3bbc2f766a1fe1b44f560c1d490220621628cbf054bdf3b80453d163a5d13cda6980d28c95c20c0133bf15135d46ec01 OP_HASH160 bdbcbb6c06a27a9746739916348f0e0775d39f18 OP_EQUAL

This is where I have questions.

  1. Rather than the non-Segwit P2SH Locking script of : OP_HASH160 <Redeem_Script_Hash> OP_EQUAL, with Segwit (P2WSH) shouldn’t the Locking Script be: 0 <Redeem_Script_Hash>
    ? And if not, why is there a mismatch if this is indeed a Segwit transaction?

  2. When executing the combined script on stack, is it run as a P2SH rather than a P2WSH? And what are the sequence of actions/executions on the stack ?

Note: I’ve looked back at P2SH stack execution, with the example there of with a dual signature MultiSig transaction. This has a quite complicated two step process of i) Check Script Hash Match and ii) Execute ‘Redeem Script’. But in this case the Unlocking Script consists of two parts, i.e. (which from the multisig example was: ‘<signatureB’> ‘<signatureC’> 2 ‘<PublicKey B’> ‘<Public C’> 3 OP_CHECKMULTISIG ) which seems quite different to what we have here, so I’m confused.

Please can you clarify and give the step-by step stack execution?

Hope this helps any Mac & Linux users reading this.

Hi @filip a little feedback.

  1. Any (decent) editor allows to search & replace (Control + H in Visual Studio Code) e.g. " with "\ instead of manually do that as you did in the video. It looks smarter and easier (and less error prone, and as such a best practice) than doing it manually.

  2. On Windows the escaping is really ugly. On Mac/Unix/Linux you can pass the transaction like in this stackexchange question answered (google was my friend to find that)

I’m a Linux user, so I used this command to run in a bash shell (the 4 variables set at the top make the script also quite easy to re-use/modify, if instead of using PowerShell in Windows, you use a bash console (should be installed when you install GIT) it may work too:

UTXO_TXID=MY_UTXO_TRXID
UTXO_VOUT=0
NEW_ADDRESS=MY_GENERATED_ADDRESS_VIA_GETNEWADDRESS
AMOUNT=49
./bitcoin-cli -regtest createrawtransaction '''
    [
      {
        "txid": "'$UTXO_TXID'",
        "vout": '$UTXO_VOUT'
      }
    ]
    ''' '''
    {
      "'$NEW_ADDRESS'": '$AMOUNT'
    }'''

See, always in linux/bash, how to assign the result of a command line to a variable: https://stackoverflow.com/questions/4651437/how-do-i-set-a-variable-to-the-output-of-a-command-in-bash

I think working with variables, when applicable, is much better (than copy/pasting stuff over and over).

1 Like

I wonder if you guys could update videos when this happens so your example works on current versions? As students, this is very confusing and could stop someone in their tracks.

1 Like

Yes, we should do that. Seems like enough have changed to motivate a re-recording. It’s on our list!!

3 Likes

Yes I agree, even notepad can do search and replace :smiley: It’s good advice, but some people that are new to programming sometimes find it easier to do things in easier but perhaps less efficient ways. Either way, I really like the cleanliness of your code. Good job!

1 Like

What is not working with the current version? I actually installed the latest version of bitcoin, “disobeying” the advise given at the beginning of the course (because, when possible I like to work with latest versions too). Also I used Linux instead of Windows. Something didn’t work out of the box, but a little search online and found the solution and everything works fine.

Hi, great questions. Sorry for the slow response. It took me some time to sit down and think it through. It was a while since I took a closer look at these types of executions. But I figured it out. What bitcoin core creates is a p2wpkh embedded in a normal p2sh. There is a lecture about this in the course. I’m not sure why they decided to do that, but I guess they have good reasons for it.

The full script you posted need to take this into account. There are two fields in the witness field, one is the signature and one is the pubkey. The redeem script is also in the tx, namely in the scriptSig field. So the full script is actually:

304402201ab25712385d15457b99a606fc22b8950a4b4f3bbc2f766a1fe1b44f560c1d490220621628cbf054bdf3b80453d163a5d13cda6980d28c95c20c0133bf15135d46ec01 0350a7dc03c4353375ac0f8cbefcf6a64b1bbb2e2d27018f687487abe45c9cf587 0 0146dfd0a493ed681fe3e7efae5d05e6b7faba8219d OP_HASH160 bdbcbb6c06a27a9746739916348f0e0775d39f18 OP_EQUAL.

If I summarize it, it looks like this:
(sig) (pubkey) (redeem script) (locking script)

The execution is rather complex, I went through an example in the lecture about this tx type. But that example was a multisig example. This is another script. I have created a pdf with the full execution of this particular script. Here is the link: https://drive.google.com/file/d/1_7ayqwa7eS7r0aX8K5k79mDlqEmdXxDX/view?usp=sharing

As I said, it was a while since I did this the last time. So there might be some small mistake somewhere, but I think it should cover the full execution pretty clearly. Let me know if you have any questions :slight_smile:

1 Like

See the above post by andersbraath1. He goes through precisely what aspects do not match the video in lengthy detail. I’m glad I found his post and Filip’s response to it. But if somebody has to dig too hard and they are a beginner they are going to get discouraged.

Thank you @filip,

Your reply is a great help and that makes sense to me now :grinning:. I’ve updated my notes accordingly!

I’ve also followed through your slides of P2SH(P2WPKH) stack execution (in the pdf) and it also makes sense to me through to the last slide (i.e. “Execute normal P2WPKH” ).

[i.e Were we have just popped the ‘redeem script’ from the stack (leaving just <pubKey> and <sig> as the two remaining elements on stack - the <pubKey> being the top element). ]

To check my understanding from this point of the remaining P2WPKH execution:
I believe that the ‘redeem script’ is actually composed of the segwit version number (0) followed by a space and then the hash of the redeem script (ie. in our example 0 0146dfd…219d ). As Bitcoin core knows this is a P2WPKH (as opposed to a P2PKH due to the format of the locking script: 0 <Pk_HASH> and its size?) it will still treat this just like a non-segwit locking script (ie. OP_DUP OP_HASH160 <PK_HASH> OP_EQUALVERIFY OP_CHECKSIG ) and execute this script.

Therefore stack execution will continue with:

  1. OP_DUP that copies the (on top of the stack) and pushes this copy onto stack.
  2. OP_HASH160 executes popping the top from stack , hashing it and pushing this <PK_HASH> onto stack.
  3. <PK_HASH> is pushed to the stack
  4. OP_EQUALVERIFY executes popping top two elements from stack (both <PK_HASH>) and as they are the same execution continues.
  5. OP_CHECKSIG -pops the last two elements from the stack (ie. < pubKey> and < sig>) to confirm that signature has been submitted by Private key that owns the Public key and if so pushed True to the stack.
  6. Therefore rightful owner has been proved (with locking script satisfying Unlocking script) and Tx will be accepted by the network.

Is the above correct?

Many thanks again for your support and assistance. All the best, Mark.

1 Like

A note for Linux Terminal users, no coma after the txid

bitcoin-cli -regtest gettxout “d500b8cdb7236510290afa8cebcf8a928b26dcd2700e1f29eb46613c2a8765d2” 0
{
“bestblock”: “31c8f03cb104b232eac813ab3229554b285b5dcdf3ce1a4a9cbf503e898e8ea9”,
“confirmations”: 103,
“value”: 50.00000000,
“scriptPubKey”: {
“asm”: “OP_HASH160 b73c579f71d10c66b880b318cc764a71afcbfa73 OP_EQUAL”,
“hex”: “a914b73c579f71d10c66b880b318cc764a71afcbfa7387”,
“reqSigs”: 1,
“type”: “scripthash”,
“addresses”: [
“2N9x5w3VcZWug2oFkKxFazt7jieoD43oh8E”
]
},
“coinbase”: true
}

1 Like

Hi Filip. I didn’t see any “generatetoaddress” part in the video, but found it here anyway.
Thanks for the great course.

1 Like

Hi @filip, what a voyage. Exactly what I signed up for!

I started this particular course with a working knowledge of Terminal [as it was] and was expecting to see $ [bash] a fair amount . .

So quite a voyage later, iTerm2, OhMyZSH, and a whole lot of other changes and learnings [and SSD upgrades, as well as fiber etc], and I’ve been running a full node 24/7 on my other machine for weeks now [up to date ofc / 0.20.0], and hoping to crack the last part of this course [the bonus parts] perhaps today. Depends on how quick I can grok package-lock json files.

It’s been quite the journey, and exactly what I needed, and was hoping for - even though you have to pick your way through the course for your particular setup - I kindof think that if you’re not comfortable doing this detective work, and compiling the knowledge for yourself outside the course, and transposing between the various terminals and across machines and so on, [I’m on a Mac generally, but am happy with my LINUX/LUBUNTU eg] then you might not really be cut out for this type of thing :slight_smile: I genuinely enjoy it.

I’ll be revisiting the last parts of this course many times, as I try to fully integrate and grok the scripting.

It’s been a great challenge, and very enjoyable.

Thanks!

3 Likes