std::shared_ptr as a last resort?

Question

I was just watching the "Going Native 2012" streams and I noticed the discussion about std::shared_ptr. I was a bit surprised to hear Bjarne's somewhat negative view on std::shared_ptr and his comment that it should be used as a "last resort" when an object's life-time is uncertain (which I believe, according to him, should infrequently be the case).

Would anyone care to explain this in a bit more depth? How can we program without std::shared_ptr and still manage object life-times in a safe way?

Answer 1

If you can avoid shared ownership then your application will be simpler and easier to understand and hence less susceptible to bugs introduced during maintenance. Complex or unclear ownership models tend to lead to difficult to follow couplings of different parts of the application through shared state that may not be easily trackable.

Given this, it is preferable to use objects with automatic storage duration and to have "value" sub-objects. Failing this, unique_ptr may be a good alternative with shared_ptr being - if not a last resort - some way down the list of desirable tools.

Answer 2

The world that Bjarne lives in is very... academic, for want of a better term. If your code can be designed and structured such that objects have very deliberate relational hierarchies, such that ownership relationships are rigid and unyielding, code flows in one direction (from high-level to low-level), and objects only talk to those lower in the hierarchy, then you won't find much need for shared_ptr. It's something you use on those rare occasions where someone has to break the rules. But otherwise, you can just stick everything in vectors or other data structures that uses value semantics, and unique_ptrs for things you have to allocate singly.

While that's a great world to live in, it's not what you get to do all of the time. If you cannot organize your code in that way, because the design of the system you're trying to make means that it is impossible (or just deeply unpleasant), then you're going to find yourself needing shared ownership of objects more and more.

In such a system, holding naked pointers is... not dangerous exactly, but it does raise questions. The great thing about shared_ptr is that it provides reasonable syntactic guarantees about the lifetime of the object. Can it be broken? Of course. But people can also const_cast things; basic care and feeding of shared_ptr should provide reasonable quality of life for allocated objects who's ownership must be shared.

Then, there are weak_ptrs, which cannot be used in the absence of a shared_ptr. If your system is rigidly structured, then you can store a naked pointer to some object, safe in the knowledge that the structure of the application ensures that the object pointed to will outlive you. You can call a function that returns a pointer to some internal or external value (find object named X, for example). In properly structured code, that function would only be available to you if the object's lifetime were guaranteed to exceed your own; thus, storing that naked pointer in your object is fine.

Since that rigidity is not always possible to achieve in real systems, you need some way to reasonably ensure the lifetime. Sometimes, you don't need full ownership; sometimes, you just need to be able to know when the pointer is bad or good. That's where weak_ptr comes in. There have been cases where I could have used a unique_ptr or boost::scoped_ptr, but I had to use a shared_ptr because I specifically needed to give someone a "volatile" pointer. A pointer who's lifetime was indeterminate, and they could query when that pointer was destroyed.

A safe way to survive when the state of the world is indeterminate.

Could that have been done by some function call to get the pointer, instead of via weak_ptr? Yes, but that could more easily be broken. A function who returns a naked pointer has no way of syntactically suggesting that the user not do something like store that pointer long-term. Returning a shared_ptr also makes it way too easy for someone to simply store it and potentially prolong the life-span of an object. Returning a weak_ptr however strongly suggests that storing the shared_ptr you get from lock is a... dubious idea. It won't stop you from doing it, but nothing in C++ stops you from breaking code. weak_ptr provides some minimal resistance from doing the natural thing.

Now, that's not to say that shared_ptr can't be overused; it certainly can. Especially pre-unique_ptr, there were many cases where I just used a boost::shared_ptr because I needed to pass a RAII pointer around or put it into a list. Without move semantics and unique_ptr, boost::shared_ptr was the only real solution.

And you can use it in places where it is quite unnecessary. As stated above, proper code structure can eliminate the need for some uses of shared_ptr. But if your system cannot be structured as such and still do what it needs to, shared_ptr will be of significant use.

Answer 3

I don't believe I've ever used std::shared_ptr.

Most of the time, an object is associated with some collection, to which it belongs for its entire lifetime. In which case you can just use whatever_collection<o_type> or whatever_collection<std::unique_ptr<o_type>>, that collection being a member of an object or an automatic variable. Of course, if you didn't need a dynamic number of objects, you could just use an automatic array of fixed-size.

Neither iteration through the collection or any other operation on the object requires a helper function to share ownership... it uses the object, then returns, and the caller guarantees that the object stays alive for the entire call. This is by far the most used contract between caller and callee.

---

Nicol Bolas commented that "If some object holds onto a naked pointer and that object dies... oops." and "Objects need to ensure that the object lives through that object's life. Only shared_ptr can do that."

I don't buy that argument. At least not that shared_ptr solves this problem. What about:

• If some hash table holds onto an object and that object's hashcode changes... oops.
• If some function is iterating a vector and an element is inserted into that vector... oops.

Like garbage collection, default use of shared_ptr encourages the programmer not to think about the contract between objects, or between function and caller. Thinking about correct preconditions and postconditions is needed, and object lifetime is just a tiny piece of that bigger pie.

Objects don't "die", some piece of code destroys them. And throwing shared_ptr at the problem instead of figuring out the call contract is a false safety.

Answer 4

I prefer not thinking in absolute terms (like "last resort") but relative to the problem domain.

C++ can offer a number of different ways to manage lifetime. Some of them try to re-conduce the objects in a stack-driven way. Some other try to escape this limitation. Some of them are "literal", some other are approximations.

Actually you can:

1. use pure value semantics. Works for relatively small objects where what is important are "values" and not "identities", where you can assume that two Person having an same name are the same person (better: two representation of a same person). Lifetime is granted by the machine stack, end -essentially- deosn't matter to the program (since a person is it s name, no matter what Person is carrying it)
2. use stack allocated objects, and related reference or pointers: allows polymorphism, and grants object lifetime. No need of "smart pointers", since you ensure no object can be "pointed" by structures that leave in the stack longer than the object they point to (first create the object, then the structures that refer to it).
3. use stack managed heap allocated objects: this is what std::vector and all containers do, and wat std::unique_ptr does (you can think to it as a vector with size 1). Again, you admit the object begin to exist (and end their existence) before (after) the data structure they refer to.

The weakness of this mehtods is that object types and quantities cannot vary during the execution of deeper stack level calls in respect to where they are created. All this techniques "fail" their strength in all the situation where creation and deletion of object are consequence of user activities, so that the runtime-type of the object is not compile-time known and there can be over-structures referring to objects the user is asking to remove from a deeper stack-level function call. In this cases, you have to either:

• introduce some discipline about managing object and related referring structures or ...
• go somehow to the dark side of "escape the pure stack based lifetime": object must leave independently of the functions that created them. And must leave ... until they're needed.

C++ isteslf doesn't have any native mechanism to monitor that event (while(are_they_needed)), hence you have to approximate with:

1. use shared ownership: objects life is bound to a "reference counter": works if "ownership" can be hierarchically organized, fails where ownership loops may exist. This is what std::shared_ptr does. And weak_ptr can be used to break the loop. This works the most of the time but fails in large design, where many designer works in different teams and there is no clear reason (something coming from a somewhat requirement) about who musty own what (the typical example are dual liked chains: is the previous owing the next referring the previous or the next owning the previous referring the next? In absebce of a requirement the tho solutions are equivalent, and in large project you risk to mix them up)
2. Use a garbage collecting heap: You simply don't care bout lifetime. You run the collector time to time and what is unreachabe is considered "not anymore needed" and ... well ... ahem ... destroyed? finalized? frozen?. There are a number of GC collector, but I never find one that is really C++ aware. The most of them free memory, not caring about object destruction.
3. Use a C++ aware garbage collector, with a proper standard methods interface. Good luck to find it.

Going to the very first solution to the last one, the amount of auxiliary data structure required to manage object lifetime increase, as the time spend to organize it and maintain it.

Garbage collector have cost, shared_ptr have less, unique_ptr even less, and stack managed objects have very few.

Is shared_ptr the "last resort"?. No, it's not: the last resort are garbage collectors. shared_ptr is actually the std:: proposed last resort. But may be the righ solution, if you are in the situation I explained.

Answer 5

The one thing mentioned by Herb Sutter in a later session is that every time you copy a shared_ptr<> there's an interlocked increment / decrement that has to happen. On multi-threaded code on a multi-core system, the memory synchronization is not insignificant. Given the choice it's better to use either a stack value, or a unique_ptr<> and pass around references or raw pointers.

Answer 6

I don't remember if last "resort" was the exact word he used, but I believe that the actual meaning of what he said was last "choice": given clear ownership conditions; unique_ptr, weak_ptr, shared_ptr and even naked pointers have their place.

One thing they all agreed upon is that we're (developers, book authors, etc) all in the "learning phase" of C++11 and patterns and styles are being defined.

As an example, Herb explained we should expect new editions of some of the seminal C++ books, such as Effective C++ (Meyers) and C++ Coding Standards (Sutter & Alexandrescu), a couple of years out while the industry's experience and best practices with C++11 pans out.

Answer 7

I think what he's getting at is that it's becoming common for everyone to write shared_ptr whenever they might have written a standard pointer (like a kind of global replacement), and that it's being used as a cop-out instead of actually designing or at least planning for object creation and deletion.

The other thing that people forget (besides the locking /updating/unlocking bottleneck mentioned in the material above), is that shared_ptr alone doesn't solve cycle problems. You can still leak resources with shared_ptr:

Object A, contains a shared pointer to another Object A Object B creates A a1 and A a2, and assigns the a1.otherA = a2; and a2.otherA = a1; Now, object B's shared pointers it used to create a1,a2 go out of scope (say at the end of a function). Now you have a leak- no one else refers to a1 and a2, but they refer to each other so their ref counts are always 1, and you've leaked.

That's the simple example, when this occurs in real code it happens usually in complicated ways. There is a solution with weak_ptr, but so many people now just do shared_ptr everywhere and don't even know of the leak problem or even of weak_ptr.

To wrap it up: I think the comments referenced by the OP boil down to this:

No matter what language you're working in (managed, unmanaged, or something in-between with reference counts like shared_ptr), you need to understand and intentionally decide on object creation, lifetimes, and destruction.

edit: even if that means "unknown, I need to use a shared_ptr," you've still thought of it and are doing so intentionally.

Answer 8

I'll answer from my experience with Objective-C, a language where all objects are reference counted and allocated on the heap. Because of having one way to treat objects, things are a lot easier for the programmer. That has allowed for standard rules to be defined that, when adhered, guarantee code robustness and no memory leaks. It also made possible for clever compiler optimizations to emerge like the recent ARC (automatic reference counting).

My point is that shared_ptr should be your first option rather than the last resort. Use reference counting by default and other options only if you are sure of what you are doing. You will be more productive and your code will be more robust.

Answer 9

I'll try to answer the question:

How can we program without std::shared_ptr and still manage object lifetimes in safe way?

C++ has large number of different ways to do memory, for example:

1. Use struct A { MyStruct s1,s2; }; instead of shared_ptr in class scope. This is only for advanced programmers because it requires that you understand how dependencies work, and requires ability to control dependencies enough to restrict them to a tree. Order of classes in header file is important aspect of this. It seems this usage is already common with native builtin c++ types, but it's use with programmer-defined classes seems to be less used because of these dependency and order of classes problems. This solution also have problems with sizeof. Programmers see problems in this as a requirement to use forward declarations or unnecessary #includes and thus many programmers will fall back to inferior solution of pointers and later to shared_ptr.
2. Use MyClass &find_obj(int i); + clone() instead of shared_ptr<MyClass> create_obj(int i);. Many programmers want to create factories for creating new objects. shared_ptr is ideally suited for this kind of usage. The problem is that it already assumes complex memory management solution using heap/free store allocation, instead of simpler stack or object based solution. Good C++ class hierarchy supports all memory management schemes, not just one of them. The reference based solution can work if the returned object is stored inside the containing object, instead of using local function scope variable. Passing ownership from factory to user code should be avoided. Copying the object after using find_obj() is good way to handle it -- normal copy-constructors and normal constructor (of different class) with refrerence parameter or clone() for polymorphic objects can handle it.
3. Use of references instead of pointers or shared_ptrs. Every c++ class has constructors, and each reference data member needs to be initialized. This usage can avoid many uses of pointers and shared_ptrs. You just need to choose if your memory is inside the object, or outside of it, and choose the struct solution or reference solution based on the decision. Problems with this solution are usually related to avoiding constructor parameters which is common but problematic practise and misunderstanding how interfaces for classes should be designed.