They grow new stems every year, so go ahead and pull out the brown (dead) one. I do that with all my lilies (day and oriental)
You can give it a tug and it it comes out easily, there ya go If not, you can leave it until it comes out by itself or you can keep testing it every so often until it does.
You are safe. Calins advice was wise, too. Bulbbils/bulblets are tiny newly formed bulbs that will grow and bloom in a few years. If you see any put them in a protected area and in a few more years you will have a few new ones bloom.If you don't see any, don't worry, they don't always make them.