FragRel: Exploiting Fragment-level Relations in the External Memory of Large Language Models