All intelligence agencies necessitate their enemies to survive. If they are eradicated, the government is unnecessary to keep their function any further. The fallout of Soviet Union deprived the western intelligence of their raison d’etre, resulted in a shrinkage of their budget. This partially holds true to the defense budget, but their capability is required as far as the country exists, not necessary to fight against the real enemy, which is a contrast to the intelligence whose prerequisites are not the same.
This is a big dilemma to the intelligence circle, as they are established to keep the country safe, but the world peace is quite unfavorable to their existence. Contrarily, they are more relied on by the administration, when the world is unstable, which incentivizes them to maintain the enemy and to show they are strong enough to spend the budget.
For example, Japanese intelligence has been necessary to show up their main enemy strong, the left extremist, which was actually quite dangerous until 1970s, but their power was declined, mainly due to the intelligence containment, though they let the extremist alive partially due to maintaining their budget and people. Furthermore, they unleash their extremist spy to sabotage, to inflate the capability of their enemies.
The next step is to widen their perimeter from the communist to other party members, including the leading Liberal Democratic Party and religious groups. This expansion is partially required to save their existence and budget, to create new enemies especially after 1990s when the communist league was collapsed. Even, the North Korean spies have been partially untouched, as they needed the enemies to prove their necessities. This change actually started from 1980s, but more deepened in 1990s, which might be supported by internal sympathizers of the left extremist, as they shared the same notion, the socialist or communist ideal.
This mutual dependency is not unique to Japan, and CIA actually needs a terrorist to maintain their existence. If the terrorist is totally eradicated, they are not wanted in this world. They were shrunk when the Soviet was fallen down, though their necessity has been recovered, as the Islamic terrorist continuously expands their grounds.
CIA did not create the terrorist, but they cannot eradicate all at once, which is partially due that it is hard to do, but partially due that it is a suicide mission they learnt in 1990s. The best situation is to have enemies, weak enough to be controlled. However, the world cannot be conspired as they want, whose failure sometimes triggers much larger problems to solve. The ISIS is one of its results.