at what layer would these value objects be instantiated? Is it in the aggregate or is it in the command handler/use case layer
Random number generators are a lot like clocks (mutable state, possibly shared, definitely not under control), so it may make sense to think about your random stream of items like time:
If you don't consider time an input value, think about it until you do - it is an important concept -- John Carmack, 1998
If that analogy works, then you'll normally do the randomness outside of the domain model (so in the "application layer", if you are using the sort of layered design that Eric Evans described in the "blue book").
This is essentially the same answer that you would get if you were following Functional Core / Imperative Shell.
So when we pass data across a boundary, it is always in the form that is most convenient for the inner circle. -- Robert Martin
Following this guideline, we'd expect the information to be expressed as a domain value prior to being passed across the boundary (so the interface of the aggregate would normally be expressed in values, not in general purpose data types).
The fact that the value happens to be randomly generated, rather than deterministically (from a user input, or a previously stored value) is an implementation detail that normally won't concern the logic of your domain model.
in what layer is underlying random value generated. Is it in these value objects themselves? Is it external to value objects?
As a general rule, your value objects will normally implement an initializer (aka a "constructor") that allows you to fix its internal information when you instantiate an instance.
So when we are talking about generating that internal information "randomly", we're normally talking about a convenience method that combines some source of entropy, possibly a data transformer / parser, and the above initializer.
Where should that convenience function live?
If all of the capabilities that you need are provided by your language and its standard library, or the core capabilities that support your domain model anyway, you could make it part of the value object.
But consider this contrived example: that you need not merely a "random" value, but one that requires cryptographically secure randomness that is not available in your standard libraries. Would you really want to introduce a new library dependency in your domain model, or would you prefer to keep such a specialized dependency out of the core of your project?
And if the answer is "we want to keep specialized dependencies out of the core", then the obvious followup question is should the location of the convenience method depend on the details of its internal representation?
My guess is that you should default to an "opt-in" policy for your convenience methods, but that such a policy needs to be balanced against other considerations (like: how much complexity does opt-in add for consumers?)
Design is what we do to get more of what we want than we would get by just doing it. -- Ruth Malan
The trick, in choosing a design, is correctly identifying the "what we want".