eryone.
Thank you for hangin' tough,
until the last session before the closing keynote,
and comin' along.
My name's Jason Clark,
I work as an engineer at New Relic,
and that informs a little bit
of what we're gonna talk about today.
I'm going to start off, and this talk's kind of sectioned
into a couple of different examples,
that you may be able to relate to.
The first one is,
has anyone here ever had a process,
they've got some Ruby program that's running,
and it's stopped.
Like it's still running, it's still there,
but it's not doing what it's supposed to do.
It's deadlocked in some fashion,
it's hung up somewhere
and you don't really know what's going on,
so you look, the process is there,
but it's hardly churnin' any CPU.
There's just no detectable activity.
You look at the log file,
and the log file's just got nothin'.
Imagine that you're in this scenario,
and just to up the ante a little,
recently at this particular place,
maybe your coworkers had been talking
about adding some multi-threading support,
into this service, you know?
It'll make it way faster, if we put some threads in there!
And all of a sudden you have this deep and abiding dread,
because you think you might have a deadlock.
Your Ruby process is stuck
and you have no idea why
and you don't know where this is happening in the code.
Well that's the sort of situation
where GDB can come to your rescue.
So GDB stands for the Gnu Debugger.
It is actually a very old tool,
the first version of it was released in 1986,
but it is a C-level debugger,
among other things there's lots of extensions to it,
but it allows you to dig into the native layer of code,
below where your Ruby is executing.
Now, we're at RubyConf,
like why am I talking about a C-debugger,
why do we care about this?
Well, the majority of us use MRI.
That's reference implementation for Ruby, Matz's Rubi,
and the truth is that Matz actually prefers
that we call it C Ruby.
The Ruby VM that you are executing your code on
is written in C.
And so GDB as a tool that has visibility to that layer
will let us see what's happening underneath.
We can take a peek at what the Ruby VM is actually doing
for us, to try to solve these sorts of problems.
So this is intended as a gentle introduction,
you're not expected to know any C,
but I'm gonna step you through
how you can use GDB as a tool to understand
what's happening in these processes underneath you.
So, with our stuck process,
there's a couple of conventions that we'll follow
in this talk and in the output.
So when there's a $ at the beginning of a line,
that indicates a command that we're gonna issue
at our terminal.
So at our terminal, we're gonna say gdb -p
and then give it the process ID
of the process that we care about,
this stuck process running on our system.
Now, GDB's gonna throw a fair amount of output.
It's gonna tell you it's attaching to things,
it may spit out stuff about symbols that it's loading,
but then eventually it lands you at this.
So when there is a (gdb) in this as well,
that is a prompt within GDB itself
where you can issue commands.
So GDB has taken the process that we were running,
it's paused its execution,
and it's now asking us,
well, what do you wanna do?
The first thing that you might want to know
is where is this code that's executing?
I seem to be stopped, where am I actually stopped at?
So, the natural thing to ask for,
is to ask GDB for a backtrace.
If you do this in actual practice,
the output may be quite a bit longer
than what I've demonstrated here.
I don't know if anyone here has ever used GDB,
but it will generate some pretty big stack traces
and a lot of output on the screen.
So some of this is lightly formatted and truncated.
But this isn't something for us to be scared of, right?
This looks a little more complicated,
but it's basically the same thing as this,
which I'm pretty sure anyone who's programmed in Ruby
for more than five minutes has seen.
This is just a stack trace,
this shows us the methods
and the files that we've stepped through.
So let's try and break down what GDB's shown us
and see where this is similar to what we know
and what we can glean out of it.
First off, because the output is pretty long
and because of some of how it displays,
it gives us these nice line numbers,
'cause some of those lines might wrap.
So those are frames that we can go through,
where we've stepped in our program,
and each of them has a function name associated with it.
You can think of these a lot like our Ruby methods.
These would be methods that you would find in your code
that you would be able to identify and locate.
In fact, a lot of the time,
if you've got the appropriate debugging symbols available,
it will even tell you
where in the source code that came from.
So here we see something in our Unix system dependencies,
but then thread.c, on the third one,
that's actually a file in MRI itself,
and this is sitting on line 4342 of that file.
So we could look and see exactly where this is executing.
And in fact, that line is very pertinent
to what we care about as well,
because we're suspecting some sort of deadlock, right?
Something that's stopping our process.
Well, MRI methods,
conventionally start with a RB underscore
in the function name,
and here we have RB underscore mutex lock.
So this is starting to smell even more like a deadlock,
like we might have expected.
When we, say, create a mutex object
and ask it to lock itself in Ruby,
this is the C function that's actually being run
by the VM on our behalf.
This is a little bit of a picture
of what's going on in this process,
but just seeing one thread being somewhere in a lock
doesn't necessarily tell us everything we need to know.
So let's ask GDB what's going on elsewhere in the process.
If we say info threads,
GDB will give us a list of all of the threads
that are in the current process,
and show us the top level
of where those are actually executing.
So the threads are numbered
and those numbers stay consistent
across the lifetime of the thread,
and like we saw in the backtrace,
it gives us the function names.
Interestingly, here we can see this pthread cond wait,
that was actually the top thing in the frame
for that particular backtrace that we were paused on,
and here we see that there are two separate threads
which are both at that same line of code,
waiting on a condition in the pthread library.
So this is giving us even more information,
this is smelling more and more like our theory
that we've got a deadlock is true, but we'd really
like to see how we got to those locations as well.
And GDB provides.
With the thread apply command,
you can select certain threads, and in this case
I'm selecting all of the threads in the process,
and ask GDB to run a specific command against those.
So we can take that backtrace that we took originally,
and that ran for just the current thread that we were on,
and instead ask GDB to show us all of the backtraces
for all of the threads.
And if we do this,
we do in fact see that thread number three
has that RB mutex lock, right where we had seen it before,
where we were paused,
and thread one is also paused at that exact same lock.
With this information in hand,
we can go to our coworker who thought they were so smart,
putting multi-treading into the thing,
point out that they may well have introduced a deadlock,
and hopefully they're smart enough to fix it.
You know, your mileage may vary depending on who it is,
and having done so, we wanna get out of GDB.
You type exit and, well,
exit's not actually defined unfortunately,
so we'll quit instead.
Let's pause for a moment before we move on
and just review a little bit
of what we've talked about here.
We've seen how you can use GDB
and attach to a running process.
You can do this to any process on your system
as long as you've got the permissions to do it.
You might not wanna do it
to something that's critical that you care about,
'cause it's easy to make things stop
and easy to make things do bad things from within GDB,
but it's a lot of fun to be able to stop things
that you want to inspect and figure out what they're doin'.
There are a number of commands
that allow us to look at the state of our process,
to understand where our code is executing,
get backtraces, and see the different things
that are going on in the life cycle and stack of our code.
And then lastly, having figured out the answer
to our question and what we needed to know,
we were able to get outta GDB
and get back to the rest of our lives.
That is a concrete example
of how you can use GDB to debug something
that's going on live on a particular process,
but I've found that GDB is also very useful
to use in exploration.
We're gonna look at what I call a unique problem,
where I saw something happening in Ruby,
and I didn't really understand
why it was doing what it was doing.
We could use GDB as the tool to dig into that,
and figure out what's happening in the VM underneath us.
The particular setup for this us the unique bang method.
Now, the unique method takes an array,
and will return to you the list of unique elements
that are in that array.
The bang methods in Ruby are mutating methods,
typically, when there is a mirroring non-bang method
available in the standard library.
It should modify the actual array in place,
so in this case that actual array that we've got there
on line one would get modified,
its members would be changed.
But as we can see here in pry,
it also returns us that list of unique elements.
This is kinda what I would expect,
it's gonna hand me back the list of unique things
when I call unique bang.
Unfortunately, if you look at it with an array
which does not have any duplication, you get nil back.
And this seems very mysterious to me.
When I first encountered it,
I was sure that something weird was going on.
So we're gonna set out,
and we're gonna go look at what's going on,
and just understand what's happening.
To do this, we're gonna set up a little script
that will duplicate this bug that we think that we've found,
this thing that we don't understand.
When we run this we see the output of the array
that's been uniqued,
and gives us the result that we expected,
and we see that our other array returns us a nil,
which was kind of confusing us.
So, how do we start this up with GDB?
As you might expect, it has some facilities
for letting you start an arbitrary process
instead of attaching to something already running.
You can do that with minus minus args,
from the command line,
so we give it the same thing that we would want to do
to run that Ruby script.
There are a variety of other ways and parameters
you can use to control the exact execution.
If you need more fine-grained control.
Like before, when GDB starts up
it's gotta drop us to a prompt,
and the program is paused.
Now, this is not a program that has run yet,
and so this is paused
before anything has happened in our program.
So to get it going, we tell GDB to go and run.
Like before, we get a lot of different output,
there's a lot of things that it's gonna tell you,
as threads spin up, as it finds debugging symbols,
but in the midst of all of that output,
we see the output to standard out,
just like we expect from our program, there in the middle.
We see our array with one two three,
and we see our nil.
So we know that this is actually running the code
that we're interested in.
It finishes that script,
lets us know that that process is finished,
and then drops us back to the GDB prompt.
We now have a process by which we can run the same thing,
we can run this test case that we wanna understand better,
over and over again, from GDB,
to try to dig into it.
But that's not as helpful as we'd like, right?
We wanna look at what's happening
specifically when we're calling this method on an array.
So we need to find some way
to coerce GDB into putting us in the location that we want.
Fortunately, the pieces are all out there
to help you very easily bridge the gap
from the Ruby code that you care about,
to the C functions that GDB understands
for us to be able to hook into.
If you go to the Ruby docs for the array class
and look at the unique bang method,
all of the documentation in Ruby
up in the upper right corner, if you hover up there,
it has a little thing that says "click to toggle source."
You don't even have to go clone the repos for Ruby
to be able to look at this information.
When you click that,
a little section expands down below
and it shows you the source code
for the particular Ruby method that you're looking at,
and so here it is, here we have that.
And don't panic, it's C.
Some of the methods you'll find will be implemented in Ruby,
things in the standard library,
but a lot of the core things in Ruby are implemented in C.
Some of you may not be familiar with that language,
some of you may be a little rusty on it,
so let's talk about one of a couple of fundamental pieces
that you need to understand
to be able to look at things in GDB and get what's going on.
This is a pretty straightforward one,
that C functions are the key to getting to the location
that you want to, to debug your problems.
So here is an example
of one of the simplest possible C functions
that you could have.
Now if you kind of squint, this looks a lot like Ruby.
But there are a few significant differences.
First off, we have to declare a return type.
We have to tell C what sort of thing we are going to return.
And if you don't return something, you have to tell it void,
for it to know that there is no return value.
We have a name for our function,
so that's very comfortable and familiar
and actually really key
to us being able to work with this in GDB.
Then we also have a list of parameters,
and because C is a somewhat type language,
these pararameters have a type.
So in this case, we have to pass a long in as the value.
C has no idea of an implicit return like we have in Ruby,
where the last line that executes
is the value that will come back.
So we have to explicitly tell C that we want to return,
and here we're just returning 42,
because it's, you know, it's the answer.
So how does this help us
with our quest to figure out what's going on
with unique bang and understand the implementation of it?
Well, if we look at that code
that showed up on our Ruby docs page,
here blown up a little bit,
we'll see some of those same pieces
that we just talked about with any C function.
So our return value, which they do a lot in MRI,
having the return value be on the line above
the name of the function,
so it returns a value,
and it takes a uppercase value as the type.
Value is the name that us used in the C code
for a Ruby object.
Any Ruby object of any type
that gets passed around throughout MRI,
will be represented by one of these values.
So here we have that unique bang takes an array,
it takes a value,
which we know is the array that we're calling this on,
and then it's going to return us back a Ruby object.
But the real key here,
what we need to be able to take our next step,
is this name.
So RB underscore array unique bang is the hook
that we need to get GDB to the location
where we can peek around
and see what's happening in the neighborhood.
As mentioned before,
RB underscore is the prefix that's used in MRI code
to indicate that things are a Ruby method.
The ARY undersore is the prefix that it follows that with
for methods that are related to the array class.
This is fairly conventional, most types in MRI that you find
will have some sort of standard naming convention
for the C functions, that maps to the Ruby representation
of when we execute this.
How do we make use of this name,
having figured out where the code is that we care about?
Well, when we ran our args command
to start GDB up with our process,
it drops us in and pauses, and in that pause,
we can tell GDB that we want to break
whenever the RB array unique bang method is called.
At that point, if we then call run,
then rather than executing the program
all the way through and terminating,
GDB will stop,
tell us that it hit a breakpoint,
and give us some of that information about it,
show us a not-very-useful-in-this-case snippet,
which is the single line of code
that it is actually paused on,
and we'll take a look at how to get a better view
around that in a moment,
and then it provides us with a prompt.
So now we are back in a paused state
where we can ask the world what it looks like,
what values are,
and control where GDB goes next.
But before we do that introspection, let's pause.
Roll back and review how we've gotten here.
To be able to run a program from scratch ourselves,
in the context of GDB,
we say GDB, minus minus args,
and then give it the invocation
just like we would do at the command line.
That'll start us in a paused state,
so we can set break points and take other operations,
and then we tell it to run,
and it'll go do the things that we wanted it to.
C functions are one of the fundamental pieces
of that language,
and they're very similar to what Ruby does with methods.
There isn't an object involved,
but they look a lot the same,
and they're the foundational way
that we can navigate around and set breakpoints within GDB.
Ruby docs, awesomely,
provides us all the information we need to translate
between the Ruby methods that we're familiar with,
and the C implementation that underlies that.
With that information, we can get ourselves to stop
anywhere that we want to in the execution of the Ruby VM,
when we're running our code.
Alright, so we've started our program,
we've gotten to the location, we've seen,
a little bit of the code around where we're at.
So let's take a deeper look at what's happening
in this unique bang method,
and what's actually around us as we're executing.
The list command,
if you have everything wired up correctly in GDB,
will show you the source code that's before and after
the line that we are paused on in our execution.
You do have to do a little bit of setup to inform GDB
where the location of a copy of Ruby's source code is,
because the compiled binaries do not have that code
inside of them, but that's pretty simple to do.
Here, we were on line 4162.
That curly brace right at the beginning of the function,
'cause we asked it to stop there for us.
So that's where we're paused,
and we can see a little bit of what's gonna come after us,
that we'll examine in a moment.
But before that, there's this array parameter.
So that's the array that we're calling this on.
And we'd really like to take a look at that.
Let's see what we can see
about what's getting passed into this function.
So GDB has a couple of methods
that are around this sort of behavior,
and have some formatting options and things.
One of them is display,
so you can say display on any local or global variable
that's in scope at the point that we're at.
And we ask to display
the array variable that's getting passed in,
and it gives us this kind of cryptic response,
it says array is equal to this very, very large number.
Well, don't panic.
But that's a memory location, and this is a pointer.
So that is the actual location in your computer's memory
where the array that's been allocated for you is residing.
Now this is probably not what you're actually wanting,
when you say display array.
You want to see what the actual values are,
or see something about how Ruby represents that.
But to get there, we need to take a slight diversion
and talk about the other important piece of C
that it's good to know at least the general shape of,
to be able to use these tools.
And that is the struct.
The struct is the way that C code allows you to group
bits of data together to carry around in your program.
So here we have a struct,
it is named Data,
so you can think of that a little like a class name,
and when we have a struct of type Data,
it has one integer in it, value.
What this means in practice is in C,
a struct is literally a description
of the layout of values in memory.
So if we have this data struct,
that means that when we make one of these structs
and place it somewhere, like at this particular location,
there will be an integer with the value that we've assigned,
sitting right at that place in memory.
Structs will nest, and you can combine them together.
So if we have a valuable data struct,
which its first member is that prior struct that we defined,
and then we put a priority after it,
what we see in memory
is we will see the integer for the value,
and then the priority from that outer containing struct
that layers there.
Now why do we care about this?
The reason that we care
is because this is how every single Ruby object
that you ever deal with in your code
is actually represented under the covers.
There is a struct called RBasic,
and RBasic has a flag,
which indicates what type of Ruby thing this is,
and a pointer to a class.
So any Ruby object, any of those value things
that we saw getting passed around,
all that you really know when you get a value,
is that at that memory location there will be the flag,
which will tell you how to interpret the rest of it,
and some data about the class.
And then depending on the type
of thing that you're looking at,
the data that follows that in memory
will take a particular shape.
For instance,
don't get scared by all of the details,
but this is what an array looks like in Ruby.
It will have the flags, the class,
and those come from the RBasic that it starts out with,
but then it has things around the length
and the capacity, and then the pointers to the values.
All of those things follow directly in memory
right after those beginning fields that we got.
Ruby has a number of different structs that it deals with,
many of the internal types,
or the basic types that you deal with in Ruby,
have explicit structs
that it implements that represent them.
So things like strings, hashes, arrays,
numbers, are all represented
by explicitly-coded structs in MRI.
But all of our Ruby objects are also there.
The RObject struct,
is the struct that is every single object
of your own custom classes
that you instantiate anywhere in your Ruby program.
So now we understand a little better,
when we said display array, what it's actually doing
is it's giving us that memory location,
where we can get a flag and a class,
and if we knew how to structure that memory,
we could interpret everything that follows
from that address, to get an actual picture
of what Ruby thinks is there.
Don't get too wigged out,
but this is a line of code that you can execute into GDB,
to tell it to actually peel that memory apart.
So GDB has loaded the RArray structure,
so it knows what fields are supposed to be there,
and it knows how to interpret that
and give us some basic display of it.
We can see in here that we've got our flags and our class,
we've got the array data itself, which looks a little funny,
but this is giving us the capability
to look literally at what the memory is
that these objects represent.
And if this was something that we knew a little more
about how arrays structured their data,
we might be able to dig into the details
and understand what's actually there.
If this was a custom object,
or you were writing a native gem, perhaps,
these might be values
that you would care about and understand.
Fortunately, Ruby provides us a simpler way
to get at these sorts of values,
with a function called RB underscore P.
This is a function that's in MRI, it's available there,
and it literally will do the sort of output printing
that you really actually want.
Like the things that you get
if you just print an object in Ruby itself.
You can invoke a C function in GDB by just saying call,
so we can call RB P and pass it our array,
and it will print out a literal representation
of that array for us.
But here we've hit a little bit of an odd spot.
Our script had an array with 1123 in it.
But the output of this array
where we hit our breakpoint is a string.
Well, I don't know exactly what's up with that,
but before we dig into this mystery,
and before we figure out
where this weird array value's coming from, let's review
what we've done in looking around the neighborhood
and looking at the values that we've got.
So if you want to see the source code
around whatever location you were paused at,
you can tell GDB to list.
And it has some parameters if you want to list more or less
of the information that's there.
To look at a local variable, you can call display on it.
And if it's just something like a C string,
or a C number, it will print that out neatly,
you might need to take a little more pain --
thanks java.
Alright.
What was I doing?
We were looking at an array.
So you can display things,
there are a lot of formatting options with display,
that you can dig into to look at the various values
that you might find around.
We've taken a brief look at C structures,
one of the fundamentals of how C programs are built
and the way that data gets passed around
for every object that you ever interact with
in your Ruby program.
And we saw the RB print,
which gives us some debug printing,
to be able to have a little bit of our Ruby niceties,
even though we're down at the C level.
But we kinda got off on a weird path here.
Like, we're seeing some values that we weren't expecting.
We're seeing this array,
this array with one string in it.
It would be really nice to know where this is coming from
and understand why we have this value.
Fortunately, MRI yet again provides us with what we need,
through the RB backtrace function.
So this function, unlike asking GDB for a backtrace,
which is the C level backtrace of where we're at,
it will give us what Ruby thinks the backtrace is,
and this gets us back
to territory that's a lot more familiar.
This answers our question pretty quickly.
If we look at the files that this is going through,
it's rubygems.
So as of any modern version of Ruby
that you will be dealing with,
rubygems is deeply integrated into Ruby itself.
And it assumes that you're gonna need some gems,
and that you're gonna do things,
and so it does some preparation
when you start running your script.
And that's what we're hitting.
Rubygems calls unique bang,
and that's the break point that we hit,
was seeing not our script running,
but rubygems calling unique bang as it started up,
before it gets anywhere near executing our script.
So knowing that, rubygems has also provided a way out,
so we can say minus minus disable gems
when we run this file,
and sure enough, when we do that,
we hit the same breakpoint,
but the array actually has the values that we expect,
it's actually running our script.
So we can start looking at the implementation
and the behavior there.
Let's look at how we can move through this function.
Just setting breakpoints everywhere is great for a start,
but we need to be able to get around.
When we listed,
we started out on the first line of this function,
on this brace.
The next two lines that are after it
are variable declarations,
and so nothing really happens on those lines,
and so when we ask GDB to go next,
much like you would in any of your Ruby debuggers,
it will move us to the next line
that has some actual activity going on it,
and it's this RB array modify check.
If you say next, like in most other debuggers,
next will continue execution
at the next line in the current function that you're in,
or return you back out,
if you've gotten to the end of the function.
Step, on the other hand, will drill in,
if there's anywhere that we can go.
In this case we were sitting on a function call,
we were getting ready to call RB modify check,
and if we say step, GDB gives us the indication
that it's moving into this other function,
tells us where it is,
and then helpfully gives us the one line
that we're actually resting on at this point in time.
If we do a listing, we can see that yes indeed,
our context has completely changed,
we're over in this totally different function
than where we were before.
If we wanna get out of that,
we can either just next through,
or you can say finish,
and that will finish the currently-executing function,
and pop you back up a level to where you were before.
Now that we've got the basics of motion,
the basics of being able to step through this code,
let's look at what the unique bang method is actually doing
and see if we can get our heads around these return values
we weren't exactly expecting.
Almost right off the bat, after we get in there,
we see this line.
So we look at the length of the array,
and if it's less than or equal to one,
we return Qnil.
Qnil is C Ruby's object
for representing nil.
That is the nil that you interact with
all the time in your Ruby applications.
That's the nill that we're all trying to escape with the
dot question mark ampersand question mark thing try,
or whatever that's coming in.
Any time you deal with a nil, that's it, right there.
But this gives us our first hint
that some of our intuition about things was probably wrong.
I mean, if the implementation here is this explicit
about returning nil in this case,
they probably really intended for us to get those nils back.
That we were kinda surprised to be seeing.
Let's keep moving on
and see a little bit more of what's around here.
If we go down, there's a mode where a block can be given,
but if we don't,
we call in to this function that says array,
make a hash out of this.
This creates us a hash
based on the values that were in the array.
The keys of that has will be the values,
so this is how unique, and unique bang,
figure out what the unique set of elements are,
in the array for us to care about.
This is actually a really interesting point,
because to me, I often will call the bang methods for things
because I figure they're more efficient in terms of memory.
They're not gonna allocate new objects,
because it's gonna modify my things in place.
But here in fact, we're allocating an entirely new hash,
which is then used just to determine
whether we've actually changed things or not later on.
So unique bang might not be getting you the savings
if you're thinking that it's avoiding memory allocations
by calling a bang method.
Having gotten that hash back,
we do the same trick that it was doing earlier,
of looking at the length of the array.
And here it says, if the length of the array
is the same as the size of the hash, we return nil.
This is the behavior that we saw up in our Ruby code,
that if there are all unique items,
if the array is already unique,
it's just gonna return you a nil.
So apparently we didn't find a Ruby bug,
but we've had an interesting time being able to dig into it.
We'll let our program continue.
You type continue, or C.
There's a lot of shortcuts as well.
Things will go on and execute from there.
And in fact,
as we go back and we're reviewing at the end of things,
apparently if we had actually read the documentation,
we might've understood that this method
was doing exactly what it was supposed to do.
But I feel like the tools are important.
Knowing that you can dig into it is worthwhile.
So in this section we've looked at moving around,
we've looked at navigating.
There's a number of other things that GDB provides,
but the documentation for it is really good,
so dig into it.
Look at your code executing live,
and see what you can see
of the underlayers of your Ruby.
To close out with,
I wanna show you just a few quick snippets
of some of the other sillier or crazier things
that you can do once you're down at this level.
One part to me is,
the output in GDB can occasionally be very confusing.
And especially if you had an application
that is also writing meaningful things to standard out.
Those things get interleaved with the debugging output,
in ways that are kind of difficult to follow.
Well, we can actually get around that.
This doesn't have anything to do, in fact, with Ruby.
This is just us interacting with C itself.
So we can tell C to close file descriptors one and two,
so that is standard out and standard error.
We can then open those,
we can open to a file,
and those file descriptors will be re-used,
and one and two will then be pointed back
to this gdb dot log file that we have.
So any of the output that our program was going to generate,
instead of going on to our screen
while we're debugging in our terminal,
will instead get sent over to a file and get saved.
This has been really useful for me
in a couple of cases where having that output
while I was actually trying to interactively debug
was very distracting and confusing.
This is probably one of the more complicated examples,
probably the most complicated in terms of commands
that we will execute here,
but it's also one of the most fun.
So we can say call, and call a C function.
Here we're calling the rp p function,
which will give us debug printing,
and what we're calling that print on
is Ruby's own eval method.
This eval method takes, a string of Ruby.
So if there is any piece of Ruby that you want to execute
while you're in the course of debugging,
you can eval it, and in this case what I'm doing
is I'm asking it to give me a backtrace print
of each of the threads.
So RB backtrace, which we looked at earlier,
is really convenient.
But it only shows you the backtrace for the current thread,
and there's no facility
for telling it to show us all of the threads.
So if we want to see the Ruby backtraces
of all of our threads, well,
we can ask Ruby to go do it itself.
Now, be aware, as with many of these things,
there is some peril.
Depending on your operating system,
if there happens to be a garbage collection run
happening at the time that your program is being paused,
this may not work properly.
But if you're already in the spot
that either you're just experimenting,
or your process is dead already,
it may be a fun thing to do.
GDB also provides us with a lot of automation facilities
and other options for controlling its behavior.
You can write a script, of GDB commands,
so here we attach to a specific process ID,
run those backtrace commands that we've looked at earlier,
and tell it, OK I'm done, go stop.
And we can run this interactively, from our command line,
saying batch and minus x,
and this will go and run a set script for us.
In fact, this was so useful,
that in the Ruby agent, we made a script called nrdebug.
This uses, among other things, it runs a GDB script,
to extract those thread backtraces.
We've used this in debugging deadlocking cases
on users' machines.
And if you install this gem, you can run nrdebug
and point it at any of your processes
and see a whole bunch of different output
that it'll provide for you,
about what's happening in that process.
This same sort of scripting
also allows you to customize your environment.
If you make a .rbinit file in your home directory,
you can put commands in there that will get executed
whenever GDB starts up,
and I can finally rest easy,
being able to type exit, instead of quit,
to get out of GDB.
Let's recap briefly.
We started off and took a look
at how GDB is a very useful tool,
in particular for debugging deadlock situations,
in places where your Ruby program might be frozen
doing something below the Ruby layer.
We looked at a unique sort of problem,
where our understanding of what the Ruby VM was doing
for us didn't match up with reality,
but we were able to look around,
we were able to move through the code,
and figure out what it was doing for us.
And realize that finding a Ruby bug
isn't always as easy as you think it might be.
Lastly, we closed out with some crazy tricks
that you can play,
and really, the sky's the limit.
We've talked about this in the context of Ruby,
but most of the applications that you're interacting with,
the libraries that are on your system,
at some point, interact with this native layer of code.
GDB can let you hook into that,
and see what your computer is really doing,
at a lower level.
I hope that you've found some cool tidbits in this today,
and I hope that maybe you'll go and debug a process
somewhere on your computer very soon.
Thank you.
(applause)
So the question is whether I found out
why unique bang returns nil,
and no, I have not gotten to the bottom
of what the rationale of it is,
but it is clearly the specified behavior of that method,
and it did bite me at one point,
that it was doing that,
as it has probably many other people in the room.
The question was,
do you need to recompile Ruby with a dash G flag,
and yes,
that is something that I did gloss over a little bit,
there may be a need to compile your Ruby
with debugging symbols,
or at a lower level of optimization.
The exact steps for doing that varies,
depending on exactly how you're installing
and compiling your Ruby,
so go look that up online.
One plug, if you do just want to play with it,
the official Docker images,
that are made for Ruby on Docker Hub,
actually have this all set up correctly and neatly,
so you can use those,
you could just spin up a little Docker container,
as separate environment,
and it has all of the things that you need,
if your locally-compiled Ruby doesn't have the symbols
that you're looking for.
The question was whether I've run into situations
where re-entrancy has been an issue in debugging,
and I don't think that I have.
There certainly are sharp edges to using a tool like this,
and it strikes me as very possible
that you could get effects like that,
which would be difficult to work around,
but I have not personally run into that.
The question was, can you disable the garbage collector
or check whether it's running
before you do some of those techniques that were hazardous,
and the answer is yes.
In fact, if you were specifically setting up a test case
for doing this sort of debugging,
unless what you're wanting to look at is GC-related,
it might be worthwhile to pause that.
There are Ruby-level methods that you can just say,
CG dot disable, and it'll turn itself off.
So that is certainly a good way in debugging
that you could handle it.
Alright, unless there's, time for one more,
but otherwise, thank you for your attention.
