I am going to give a more detailed example of how to use pre/post conditions and invariants to develop a correct loop. Together such assertions are called a specification or contract.
I am not suggesting that you try to do this for every loop. But I hope that you will find it useful to see the thought process involved.
In order to do so, I will translate your method into a tool called Microsoft Dafny, which is designed to prove the correctness of such specifications. It also checks termination of each loop. Please note that Dafny does not have a for loop so I have had to use a while loop instead.
Finally I will show how you can use such specifications to design an, arguably, slightly simpler version of your loop. This simpler loop version does infact have the loop condition j > 0 and the assignment array[j] = value - as was your initial intuition.
Dafny will prove for us that both of these loops are correct and do the same thing.
I will then make a general claim, based on my experience, about how to write correct backwards loop, that will perhaps help you if faced with this situation in the future.
Part One - Writing a specification for the method
The first challenge we are faced with is determining what the method is actually supposed to do. To this end I designed pre and post conditions which specify the behaviour of the method. To make the specification more exact I have enhanced the method to make it return the index where value was inserted.
method insert(arr:array<int>, rightIndex:int, value:int) returns (index:int)
// the method will modify the array
modifies arr
// the array will not be null
requires arr != null
// the right index is within the bounds of the array
// but not the last item
requires 0 <= rightIndex < arr.Length - 1
// value will be inserted into the array at index
ensures arr[index] == value
// index is within the bounds of the array
ensures 0 <= index <= rightIndex + 1
// the array to the left of index is not modified
ensures arr[..index] == old(arr[..index])
// the array to the right of index, up to right index is
// shifted to the right by one place
ensures arr[index+1..rightIndex+2] == old(arr[index..rightIndex+1])
// the array to the right of rightIndex+1 is not modified
ensures arr[rightIndex+2..] == old(arr[rightIndex+2..])
This specification fully captures the behaviour of the method. My main observation about this specification is that it would be simplified if the procedure was passed the value rightIndex+1 rather than rightIndex. But since I cannot see where this method is called from I do not know what effect that change would have on the rest of the program.
Part Two - determining a loop invariant
Now we have a specification for the behaviour of the method, we have to add a specification of the loop behaviour that will convince Dafny that executing the loop will terminate and will result in the desired final state of array.
The following is your original loop, translated into Dafny syntax with loop invariants added. I have also changed it to return the index where value was inserted.
{
// take a copy of the initial array, so we can refer to it later
// ghost variables do not affect program execution, they are just
// for specification
ghost var initialArr := arr[..];
var j := rightIndex;
while(j >= 0 && arr[j] > value)
// the loop always decreases j, so it will terminate
decreases j
// j remains within the loop index off-by-one
invariant -1 <= j < arr.Length
// the right side of the array is not modified
invariant arr[rightIndex+2..] == initialArr[rightIndex+2..]
// the part of the array looked at by the loop so far is
// shifted by one place to the right
invariant arr[j+2..rightIndex+2] == initialArr[j+1..rightIndex+1]
// the part of the array not looked at yet is not modified
invariant arr[..j+1] == initialArr[..j+1]
{
arr[j + 1] := arr[j];
j := j-1;
}
arr[j + 1] := value;
return j+1; // return the position of the insert
}
This verifies in Dafny. You can see it yourself by following this link. So your loop does correctly implement the method specification that I wrote in part one. You will need to decide if this method specification is really the behaviour that you wanted.
Note that Dafny is producing a proof of correctness here. This is a far stronger guarantee of correctness than can possibly be obtained by testing.
Part Three - a simpler loop
Now that we have a method specification that captures the behaviour of the loop. We can safely modify the implementation of the loop while still retaining confidence that we have not changed the loop behaviour.
I have modified the loop so that it matches your original intuitions about the loop condition and final value of j. I would argue that this loop is simpler than the loop you described in your question. It is more often able to use j rather than j+1.
Start j at rightIndex+1
Change the loop condition to j > 0 && arr[j-1] > value
Change the assignment to arr[j] := value
Decrement the loop counter at the end of the loop rather than the begining
Here is the code. Note that the loop invariants are also somewhat easier to write now:
method insert2(arr:array<int>, rightIndex:int, value:int) returns (index:int)
modifies arr
requires arr != null
requires 0 <= rightIndex < arr.Length - 1
ensures 0 <= index <= rightIndex + 1
ensures arr[..index] == old(arr[..index])
ensures arr[index] == value
ensures arr[index+1..rightIndex+2] == old(arr[index..rightIndex+1])
ensures arr[rightIndex+2..] == old(arr[rightIndex+2..])
{
ghost var initialArr := arr[..];
var j := rightIndex+1;
while(j > 0 && arr[j-1] > value)
decreases j
invariant 0 <= j <= arr.Length
invariant arr[rightIndex+2..] == initialArr[rightIndex+2..]
invariant arr[j+1..rightIndex+2] == initialArr[j..rightIndex+1]
invariant arr[..j] == initialArr[..j]
{
j := j-1;
arr[j + 1] := arr[j];
}
arr[j] := value;
return j;
}
Part Four - advice about backward looping
After having written and proved correct many loops over quite a few years, I have the following general advice about looping backwards.
It is almost always easier to think about and write a backward (decrementing) loop if the decrement is performed at the beginning of the loop rather than the end.
Unfortunately the for loop construct in many languages makes this difficult.
I suspect (but cannot prove) that this complexity is what caused the difference in your intuition about what the loop should be and what it actually needed to be. You are used to thinking about forward (incrementing) loops. When you want to write a backward (decrementing) loop you try to create the loop by trying to reverse the order that things happen in a forward (incrementing) loop. But because of the way the for construct works you neglected to reverse the order of the assignment and loop variable update - which is needed for a true reversal of the order of operations between a backward and forward loop.
Part Five - bonus
Just for completeness, here is the code you get if you pass rightIndex+1 to the method rather than rightIndex. This changes eliminates all the +2 offsets that are otherwise required to think about the correctness of the loop.
method insert3(arr:array<int>, rightIndex:int, value:int) returns (index:int)
modifies arr
requires arr != null
requires 1 <= rightIndex < arr.Length
ensures 0 <= index <= rightIndex
ensures arr[..index] == old(arr[..index])
ensures arr[index] == value
ensures arr[index+1..rightIndex+1] == old(arr[index..rightIndex])
ensures arr[rightIndex+1..] == old(arr[rightIndex+1..])
{
ghost var initialArr := arr[..];
var j := rightIndex;
while(j > 0 && arr[j-1] > value)
decreases j
invariant 0 <= j <= arr.Length
invariant arr[rightIndex+1..] == initialArr[rightIndex+1..]
invariant arr[j+1..rightIndex+1] == initialArr[j..rightIndex]
invariant arr[..j] == initialArr[..j]
{
j := j-1;
arr[j + 1] := arr[j];
}
arr[j] := value;
return j;
}
