When a dir has no delete permission, the draggable isn't initialized on files. This fix makes sure we don't try to destroy a draggable when it wasn't inited in the first place. Fixes #6254
