How the Taliban Outwitted and Outwaited the U.S.

The Taliban won’t seek to rule Afghanistan on their own anymore, the document assured, and a new constitution “would pave the way for power-sharing in the next government.” When the republic’s delegates returned to Kabul, many enthused about how much the Taliban had evolved from the ruthless regime that ruled Afghanistan in the 1990s.

For the next nine years, the Taliban continued to lull the world with conciliatory messaging as they pursued a bloody war at home in parallel with diplomatic efforts to secure their ultimate goal: an American military withdrawal.

“Monopoly of power is a story of failure. That is why we want to have all on board,” Suhail Shaheen, now the Taliban’s ambassador-designate to the United Nations, said in an interview with The Wall Street Journal six weeks before the group seized Kabul, deposed the Afghan republic and monopolized all power. “Past experiences have shown that you will ultimately fail and will not bring durable peace.”

Throughout its history, Afghanistan defied foreign attempts to reshape the country, from the British Empire in the 19th century to the Soviet occupation in the 1980s to the failed American experiment in nation-building.

An examination of why U.S. peace efforts collapsed so spectacularly, setting back the Biden presidency and America’s global standing, reveals the Taliban’s mastery of the diplomatic long game.

America’s increasing impatience with its longest overseas war drove the pace of these talks—removing one by one the Taliban’s incentives to compromise. For President Biden just as for President Trump, the “priority was to get out, not the Afghan settlement,” said Zalmay Khalilzad, who served as chief U.S. negotiator under both administrations. “They made it clear—and that strengthened the Talibs.”

Seeking an exit, U.S. officials found it expedient to paint Taliban behavior in the best possible light while exaggerating the strength of the Afghan republic they had brought to life. Recognizing this opening, the Taliban leadership learned how to obfuscate their true intentions in the comforting language that appealed to foreign diplomats and negotiators.

The question now is whether Western powers can apply lessons from past failures as they try to nudge the Islamist movement into adopting more-moderate policies. Experience suggests that the Taliban won’t readily trade long-held traditions for Western cash and a place in the global community.

Some U.S. and former Afghan officials continue to believe the relatively pragmatic Taliban they dealt with were sincere and that a negotiated solution could have preserved at least some achievements gleaned from the 20-year international effort in Afghanistan. Intransigence by President Ashraf Ghani, they argue, ultimately torpedoed these efforts and bolstered the Taliban’s more hard-line elements.

Unable to fight once American support disappeared, Afghanistan’s armed forces disintegrated in August, allowing the Taliban to seize almost all of the country’s provincial capitals and reach the outskirts of Kabul in just over a week. The collapse of remaining government structures after Mr. Ghani fled the country on Aug. 15 rendered U.S.-backed talks on a peaceful transition moot.

The new Afghan government established in September is made up almost exclusively of Taliban clerics prominent in the insurgency. While the new regime has refrained so far from openly hosting terrorist groups or committing the kind of atrocities that earned it world-wide condemnation in the past, it has already sharply curtailed the rights of women, banned girls’ education beyond the sixth grade in most provinces and marginalized ethnic communities that aren’t part of its Pashtun power base.

In continuing talks with U.S. and allies in Doha, Qatar, the new Taliban administration is seeking diplomatic recognition, a removal of American sanctions and the unfreezing of over $9 billion in Afghan central-bank assets abroad. One of Washington’s key conditions is the creation of a more inclusive government in Kabul that respects human rights, one that would fulfill promises that the Taliban have been making since Chantilly.

“The Taliban regime should seek legitimacy within Afghanistan before seeking international recognition,” said Thomas West, the U.S. special representative for Afghanistan, who is leading these talks.

The Road to Doha

The Taliban sought to negotiate with Washington and other Afghans immediately after a U.S. invasion ousted their government in 2001. Hamid Karzai, Afghanistan’s American-anointed new leader, wanted the Islamist movement to participate in the Bonn conference that year that established the country’s new political order. Washington, still shaken in the immediate aftermath of the Sept. 11 attacks, which Osama bin Laden plotted on Afghan soil, vetoed the plan. Potential Taliban negotiators were hunted down by U.S. special-operations forces and the Central Intelligence Agency, and shipped to detention in Guantánamo Bay, Cuba.

American and allied attitudes to engaging the Taliban changed as the group bounced back in the ensuing decade. By 2009, the Taliban once again controlled large parts of the countryside. Mr. Obama surged the U.S. military presence to over 100,000 troops to defend the Afghan republic—while also promising to start withdrawing all American forces 18 months later.

By the time Washington was ready to negotiate, Taliban leaders refused to sit down with Mr. Karzai’s administration, dismissing it as an American puppet with no legitimacy or agency of its own. Mr. Karzai, for his part, objected to the U.S. engaging in talks with the Taliban that excluded the Afghan republic’s democratically elected government. The Obama administration agreed not to discuss Afghanistan’s future without Kabul but also endorsed the idea of creating a Taliban political mission abroad to facilitate diplomatic contacts.

The U.S. and the insurgents began building trust by negotiating tactical deals, such as freeing five senior Taliban leaders who had spent more than a decade in Guantánamo in exchange for the Taliban handing over Bowe Bergdahl, a U.S. Army sergeant who walked off his base and was captured by the insurgents. Taliban representatives, some of whom had been living in Doha for years, formally opened a political office there in 2013.

While the Taliban still rejected direct talks with the Kabul government, its envoys based in Doha began to engage in several rounds of so-called track-two meetings with members of the Afghan republic’s political elites. The Chantilly confab was followed by similar events in Europe, Russia and China.

Over the years, the Taliban office in Doha, and the exemption of its members from United Nations travel sanctions, allowed the insurgent movement to reach out to governments world-wide, gaining growing acceptance as a legitimate political force.

“One of the reasons why the Taliban outsmarted Americans is the fact they set up relations with the whole world while negotiating with the Americans—something that the Americans didn’t want to happen,” said Rahimullah Mahmood, a veteran insurgent commander who served as governor of Wardak province after the Taliban takeover and now is deputy head of the Kandahar-based military corps. “They succeeded in convincing the world that the Taliban weren’t the terrorists as depicted by American propaganda.”

In 2018, President Trump, a longtime critic of the Afghan war, scrapped the long-held precondition that the U.S. would only enter into talks with the Taliban that included the Afghan republic’s government. Mr. Khalilzad, a former U.S. ambassador to Kabul and to the United Nations, was appointed as special envoy with wide latitude to negotiate a deal.

Born in Afghanistan in 1951, Mr. Khalilzad knew Mr. Ghani since both went to the U.S. as high-school exchange students. The two men later studied at the American University in Beirut and then earned their Ph.D.s in the U.S.—Mr. Khalilzad at the University of Chicago, and Mr. Ghani at Columbia. Mr. Khalilzad’s dealings with the Taliban dated back to the 1990s, when he served as a consultant for the Unocal oil company that explored building a pipeline through Afghanistan.

“His mandate was to figure out a way to enable us to leave quickly and potentially zero out the force, but to be able to call it a victory,” said a senior State Department official who was involved in the effort. “And it wasn’t always understood that those were mostly mutually exclusive.”

Mr. Ghani, a former American citizen who succeeded Mr. Karzai as president in 2014, was alarmed by these negotiations. A co-author of a book called “Fixing Failed States” and a onetime fixture of Washington’s think-tank circuit, he boasted to other Afghan officials about his understanding of American politics. But, until too late, he and senior officials in his administration misread American intentions and clung on to illusions that Washington would never actually pull the plug on Kabul.

The U.S. had been talking about leaving Afghanistan for more than a decade, after all. “There was this notion of Afghanistan being a unique geographical location that would always be an area of interest for global powers,” said Nader Nadery, a senior Afghan peace negotiator who headed the fallen republic’s civil service. “Some of our colleagues believed until the last months that the U.S. forces would never leave.”

“In Kabul, they were living in an unrealistic world,” agreed Mr. Khalilzad, who left the U.S. government in October. “That was the grand miscalculation.”

That belief that America’s national-security establishment wouldn’t allow Mr. Trump or Mr. Biden to abandon Afghanistan was coupled with another strategic blunder: excessive optimism about the Afghan republic’s own military strength, Mr. Khalilzad added. “They didn’t assess their forces correctly. I don’t know that any of them thought, at the leadership level, that the force would collapse that quickly.”

The combination of these two miscalculations meant that Mr. Ghani slow-rolled peace talks between the Afghan republic and the Taliban on a possible power-sharing agreement that would have inevitably involved him leaving office. It is unclear to what extent the Taliban would have compromised. But, as the insurgents made dramatic military gains, their calculations changed, too. In Doha over the months, discussions moved from possible power-sharing to considering an “inclusive government” dominated by the Taliban to essentially a surrender on Taliban terms.

“Ghani was not flexible, and that is why we are in this dark situation,” said Habiba Sarabi, a member of the Afghan republic’s negotiating team with the Taliban and a former governor of Bamian province. “His mentality was that the Taliban should join his government and he would be on the top. This was not possible in a peace process. He loved power. He was crazy for power.”

Ms. Sarabi, who like most of the Afghan republic’s senior officials and negotiators is now in exile, added that Mr. Khalilzad shared the blame because he consistently stressed the Taliban’s alleged moderation and interest in a peaceful transition. “He wanted to sugarcoat the almond. But at the end the bitter taste appeared,” she said.

Mr. Khalilzad, who wrote an op-ed all the way back in 1996 to argue that “the Taliban does not practice the anti-U.S. style of fundamentalism,” said that he believed in the sincerity of Taliban negotiators and that it was the fault of both sides that no political settlement could be found. “They didn’t rise to the occasion,” he said. “I couldn’t blame that one side was more at fault than the other.”

Withdrawal or Peace?

To begin serious talks, Mr. Khalilzad needed a Taliban counterpart with appropriate seniority. Mullah Abdul Ghani Baradar fit the bill. He was a co-founder of the Islamist organization, served as deputy minister of defense in the previous Taliban regime and coordinated the insurgency’s commanders after the U.S. invasion. A relative pragmatist, Mr. Baradar had tried to open negotiations with the U.S. in 2001, and engaged in secret contacts with Mr. Karzai’s government in 2010. One of the few senior Taliban members from the same aristocratic Popolzai clan as Mr. Karzai, Mr. Baradar was captured by Pakistani and U.S. agents in Karachi later that year, and kept in Pakistani custody since.

In September 2018, Secretary of State Mike Pompeo led a delegation to Islamabad to press the need for Pakistan’s cooperation and to demand Mr. Baradar’s release. Pakistan acquiesced and Mr. Baradar moved to Doha weeks later to take the helm of the Taliban political office. The Taliban’s secretive supreme leader Hibatullah Akhundzada, who has never been filmed in public, gave his blessing to the negotiations.

The talks faced a constraint from the start: Mr. Trump’s impatience to bring home the troops. American negotiators say they woke up every morning with the fear of seeing what they described as “the tweet of Damocles” in which Mr. Trump would announce an unconditional withdrawal.

As American and Taliban envoys started hashing out a deal in Doha, U.S. ambassador to Kabul John Bass tried for months to push Mr. Ghani to name a broad negotiating team that would be ready to begin Kabul’s own talks with the Taliban. The Afghan president refused, unwilling to dilute his administration’s control over the process.

“President Ghani’s model of negotiation—and that was the essence of his unhappiness—was that he should be the one negotiating with Hibatullah. That he would have his laptop under his arm, sit with Hibatullah, and make a deal,” Mr. Khalilzad said. “And of course that was not realistic from the get-go.”

By the summer of 2019, Mr. Khalilzad’s team hammered out the broad contours of the deal with Mr. Baradar in Qatar. Then, the Taliban suddenly reversed course and demanded prisoner releases, a new, major concession. To break the deadlock, the U.S. yielded and signed off on a clause that required Kabul to free up to 5,000 Taliban prisoners in Afghan custody. Mr. Ghani was allowed to read the draft text but not to keep a copy. He wasn’t given access to the agreement’s secret annexes, either.

With preparations under way for Mr. Trump to host a grand signing ceremony around the Sept. 11 anniversary, a car bomb went off near the U.S. Embassy and Afghan security compounds in Kabul, killing 12 people, including a U.S. soldier. The Taliban claimed responsibility. A furious Mr. Trump tweeted that he “called off” the talks with the Islamist movement and canceled plans for a meeting with Taliban leaders and Mr. Ghani in Camp David.

Encouraged by the apparent about-turn, Mr. Ghani hoped that Mr. Trump’s rush for the exits would now be restrained. His national-security adviser, Hamdullah Mohib, complained that America was “whitewashing the Taliban” because it was tired of the war, and called for reassessing the deal. Mr. Nadery, the peace negotiator, wasn’t as optimistic. That September, he binge-watched a Netflix series on the fall of South Vietnam, noting that the government in Saigon, just as the government in Kabul, had been kept in the dark by the U.S.

In Washington, John Bolton, Mr. Trump’s then-national security adviser, held a similar view. “We were basically selling the government out. The analogy of Vietnam is really true,” said Mr. Bolton, who quit that month over disagreements with Mr. Trump that included Afghanistan policy. “In both cases, everybody, every other interested party could see that the principal U.S. objective was to get out.”

The suspension didn’t last long. Mr. Trump still wanted to leave Afghanistan before the U.S. presidential elections. Within weeks, U.S. diplomats opened talks to swap two professors of the American University in Kabul held hostage by the Taliban in return for Anas Haqqani, the younger brother of the Taliban’s deputy leader Sirajuddin, who was held by the Afghan government. The U.S. has designated the Haqqani network a terrorist organization since 2012 because of its links to al Qaeda.

By February 2020, the Taliban agreed to a brief cease-fire as a show of goodwill and Mr. Trump approved signing the deal. It was officially called the “Agreement for Bringing Peace to Afghanistan,” even though the Taliban made no commitment to stop military operations against the Afghan government and security forces.

In the text, the U.S. promised a full military withdrawal by May 2021 in exchange for the Taliban pledging to prevent terrorist groups from using Afghan soil to threaten other nations. The Taliban, in a significant departure, also agreed to open peace talks with Mr. Ghani’s government. The U.S. withdrawal wasn’t conditional on the success of these negotiations—in part because Washington didn’t want to give Mr. Ghani a lever to slow down the departure.

Mr. Pompeo flew to Doha to attend the signing ceremony on Feb. 29, 2020. Minutes before his arrival in Qatar, the Taliban staged a victory march with the white flags of their Islamic Emirate, prompting fears among the Qatari hosts that the embarrassment might scuttle the deal at the last moment. The Qataris were prepared to prevent the Taliban from entering the luxury Sheraton resort with the flags. The insurgency’s representatives left them in their vehicles.

Mr. Pompeo grimly shook hands with Mr. Baradar after aides failed to orchestrate his separation from the Taliban in the room. Mr. Khalilzad signed for the U.S. while Mr. Pompeo followed with a somber speech delivered mostly to journalists in another room afterward. Members of Mr. Khalilzad’s team were relieved the day had passed without incident and stayed out until late in Doha, drinking overpriced cocktails.

Mr. Ghani initially resisted the Doha agreement’s commitment, made by the U.S. without his assent, that Kabul release thousands of Taliban prisoners. He also kept rebuffing American pressure to create a negotiating team including his political foes in Kabul, such as Mr. Karzai and his challenger in the 2019 presidential elections, Abdullah Abdullah. Any power-sharing deal with the insurgents would be contingent on Mr. Ghani stepping down, after all. Loath to leave office, the Afghan president instead kept hoping that Washington would reverse the withdrawal decision, especially if Mr. Trump were to fail in his re-election bid.

“We, the Afghan government, should have seen the writing on the wall,” Mr. Mohib, who served as Mr. Ghani’s national-security adviser until both men fled Kabul on Aug. 15, said when asked what was the Afghan administration’s biggest error. “It was a withdrawal, not a peace agreement. Democratic values were not as much of a priority as we thought. The gains of the past 20 years were not as much of a priority as we thought they would be.”

Taliban military commanders were also initially upset with the Doha deal. Mullah Mohammad Fazel, a Taliban negotiator and one of the five former Guantánamo inmates freed in exchange for Sgt. Bergdahl, traveled across front lines from Qatar to a meeting with insurgent commanders from all over Afghanistan to explain its terms.

Some of the men, sporting the Taliban’s black turbans and beards, believed the agreement was naive, according to those present. How were they supposed to trust that the U.S. would in fact leave Afghanistan the following year? Why should they stop hitting American forces even as Washington retained the right to conduct airstrikes against them?

“During the negotiations, many were claiming that the Americans were deceiving us, that it was all a trap for us,” said Mr. Mahmood, then the military commander of the Taliban’s eastern zone, who attended the gathering in the Musa Qala district of Helmand province. “Many military commanders wanted to resume attacks on Americans. The suicide bombers, in particular, were extremely sad: they cried and mourned the fact that they wouldn’t get martyred.”

Yet, the Taliban political negotiators’ argument that Washington would deliver on pledges made in Doha and withdraw from Afghanistan prevailed at the end, said Mr. Mahmood. “It’s a treaty of victory,” was the message that he carried back to his troops.

Shortly after that, the Taliban’s propaganda department published a calendar for the Islamic year 1442 that began in August 2020. It showed an American and a Taliban hand signing the Doha deal—described as “the agreement to end the invasion”—and Afghanistan breaking free from chains of foreign occupation. Below was a quote from the Taliban’s supreme leader, Mr. Hibatullah, pledging: “We don’t want the monopoly on power.”

Drawdown

The lack of progress in Afghanistan ahead of the U.S. presidential election was causing Mr. Trump to get impatient, and in June he ordered a fresh drawdown of troops to 4,500, without any concessions by the Taliban.

At that point, the Taliban hadn’t delivered on any of their major promises except for stopping attacks on American troops. They still refused to meet the Afghan government’s delegation. Trying to gain the prisoner release and break the stalemate, Mr. Baradar made verbal assurances to U.S. negotiators that violence would drop as soon as the 5,000 Taliban inmates were set free.

A buoyed Mr. Khalilzad sent a cable to Washington announcing that Mr. Baradar had promised a near-complete cease-fire. Ross Wilson, who had taken over the role of top U.S. diplomat in Kabul, delivered the message to Mr. Ghani. The promised cease-fire “was part of our selling of what was a very difficult decision for good reasons,” Mr. Wilson said. Grudgingly, Mr. Ghani agreed to a prisoner release in phases in exchange for the Taliban setting free 1,000 government personnel in their custody.

With the release complete in September 2020, Taliban and Afghan republic negotiators finally gathered in Doha’s Sharq Village resort for their own peace talks. The venue spread around a large beachside pool frequented by bikini-clad tourists who lounged under loud pop music that wafted into Taliban negotiators’ rooms. Afghan republic delegates were told by Kabul to stay away from the pool to avoid embarrassing headlines. The Taliban didn’t swim.

The two sides had breakfast in separate halls and rarely socialized. Key Taliban negotiators, who by then spent several years in Qatar and had families and businesses there, only occasionally showed up in the Sharq Village.

As the two Afghan delegations began their discussions, a U.S. military team monitored the levels of violence in Afghanistan to evaluate whether the Taliban were abiding by Mr. Baradar’s assurances. The team documented a rise in insurgent attacks instead. U.S. Army Col. Brad Moses, who served as deputy to the U.S. military commander in Afghanistan, Gen. Scott Miller, briefed about the alarming data on intensifying violence during regular calls with the White House, the State Department, the CIA and other U.S. government agencies.

“It never reduced,” he said. The Taliban would claim to the U.S. that these attacks were either carried out by spoilers or criminals when confronted with the evidence, he added.

The Afghan government, meanwhile, instructed its forces, cooped up in isolated bases and outposts, to stop offensive operations during the talks and engage in what it called “active defense.” The loss of initiative handed over a critical advantage to the insurgents, said Lt. Gen. Imam Nazar Behboud, who commanded the Afghan army’s Kandahar corps.

“This meant that you just had to stand there and wait until the Taliban attacked you. No matter how much you got killed, you just had to wait,” he said. “There were huge casualties. The troops were tired, they were not receiving any backup from Kabul, and they lost their trust in the central government.”

By October, the Taliban had gathered a huge force in the south and launched a wide-scale assault on Helmand’s provincial capital of Lashkar Gah. The U.S. intervened with airstrikes to prevent the city’s collapse. Weeks later, the Taliban moved toward Kandahar, capturing the Arghandab district on the edge of the country’s second-largest city. Another torrent of U.S. airstrikes stopped further advances. Both sides accused each other of violating the Doha agreement.

Still, the Taliban stuck to their promise not to strike American targets, showing that they could exercise discipline over their fighters when they wanted to. Despite sustaining heavy casualties in the airstrikes, the Taliban leaders calculated it wasn’t in their interest to disrupt an American withdrawal they viewed as inevitable.

“We convinced our fighters that, as our negotiations with the Americans are under way, we will not fire a single bullet at the Americans. We proved that we can uphold our treaties,” said Mohammad Farouk Ansari, a member of the Taliban’s military commission that united some 50 top commanders from across the country. “We told each other at the time that it was a victory. When the Americans started closing their outposts and evacuating their bases, we knew that the country was ours, today or tomorrow.”

U.S. officials still wonder whether they had been played by Mr. Baradar’s promises or whether the chief Taliban negotiator himself was being used by the insurgency’s real leadership to lull the U.S. and Kabul into complacency.

‘It was always hard to tell if the Taliban were serious about a political settlement or not,” said Carter Malkasian, who was part of Mr. Khalilzad’s team as a representative from the Joint Chiefs of Staff. “One possibility is that they never meant it. That they were saying what we needed to hear. We may learn, like we have about the Vietnamese negotiations, that they never had any intention of conceding.”

The U.S. presidential election was held on Nov. 3 and Mr. Trump lost. While fighting to overturn the results, he ordered the Pentagon to pull remaining troops out of Afghanistan and appointed a new defense secretary, Chris Miller, a former Green Beret and vocal war skeptic, to carry out the plan. Mr. Miller, along with other close advisers, convinced the president to keep a downsized force of 2,500 troops in Afghanistan to avoid the country’s collapse, which they said would hurt Mr. Trump if he wanted to run for office again.

Around that time, Mr. Khalilzad circulated proposals for a new interim government that would be equally split between the Taliban and representatives of the republic. The proposal, he said, didn’t specify who would be in charge.

Mr. Miller said the unspoken goal of retaining a small force to keep the Kabul government afloat was to eventually force Mr. Ghani to cut a power-sharing deal. “And let’s be honest, the Taliban probably would have had about 14 seats in the cabinet. And Ghani probably would have had four. He probably would have had sports and recreation. Probably would have had, like, roads and sewers,” Mr. Miller added.

The Afghan president hoped the American determination to withdraw from Afghanistan would end with Mr. Trump’s term on Jan. 20. He was so convinced that the new Biden administration wouldn’t follow through on the Doha agreement that he declined to see Mr. Khalilzad when the American envoy came to Afghanistan that January. Mr. Ghani subsequently rejected Mr. Khalilzad’s power-sharing plan, which was promptly leaked to the media, and kept refusing to engage in meaningful talks in Doha.

“It was us, the republic, that were lingering. The Taliban were much more flexible,” said Fatima Gailani, a negotiator for the republic who belongs to one of the country’s most influential families. “Negotiations need a give and take, and an honorable compromise is absolutely fine, but that was not the case at all. It was purposefully lingering and waiting for Biden to come. Why were they thinking that Biden would bring a miracle, I don’t know.”

Mr. Khalilzad gave his proposal to Mr. Baradar, who agreed to consider it but offered no formal response.

By then, Taliban commanders on the ground, emboldened by their military successes and the looming American withdrawal, had little desire to share power with their enemies. “The strategy of a colonizer, when it is forced out of a country, is to leave its offspring behind, so as not to break the chain of colonization. The Americans wanted to keep a parallel government here, for the Taliban and the rest to have equal power,” said Mr. Ansari, the Taliban military commission member who operated southeast of Kabul. “We did not agree with this from the very beginning. We said that we’re the rulers in the country. The country is our home. We don’t accept a second ruler in our home.”

Mr. Ghani’s hopes about Mr. Biden were quickly dashed. The new president had advocated withdrawing from Afghanistan back when he served as Mr. Obama’s vice president, and showed little inclination to reverse Mr. Trump’s deal.

For months after Mr. Biden took office, interagency officials held an endless series of meetings on how to mitigate risks from the pullout. Abandoning the Doha agreement, the White House calculated, would force the Taliban to resume attacks on American forces, requiring a major troop increase with no end in sight. As for the peace talks shepherded by Mr. Khalilzad in Doha, the White House concluded that chances of progress were too slim to justify delaying the withdrawal.

“There is not a lot of evidence that either side treated those negotiations in Doha in good faith,” said a current senior Biden administration official who was involved in the decision-making.

On April 12, the Taliban refused to participate in a peace conference that the U.S. was trying to convene under the sponsorship of the United Nations in Turkey, fearing that they would be forced to make concessions.

Two days later, Mr. Biden announced that all U.S. forces would be out of Afghanistan by Sept. 11, regardless of whether the Taliban and the Afghan reach a political deal or any other developments on the ground, a move that removed the conditionality attached to the 2020 Doha agreement.

“We will not conduct a hasty rush to the exit. We’ll do it…responsibly, deliberately and safely,” Mr. Biden said in the White House’s Treaty Room that day. “More and endless American military force could not create or sustain a durable Afghan government.”

Kabul was stunned. The following afternoon, Mr. Ghani convened top Afghan security officials to discuss Mr. Biden’s bombshell. The army chief of staff wondered how the Afghan military could continue servicing its aircraft once American advisers and contractors left. Mr. Ghani, according to a person present at the meeting, was calm and said he was working on securing continuing U.S. support.

Vice President Amrullah Saleh, who used to work closely with the CIA, refused to believe that Mr. Biden would actually withdraw all U.S. forces. Could Mr. Biden’s announcement simply be a pressure tactic to force Kabul make concessions to the Taliban in Doha, he wondered, according to people present.

Mr. Saleh, Afghanistan’s former intelligence chief, told the Journal his U.S. interlocutors had been assuring until the last moment that Washington wouldn’t abandon his administration. “There were so many occasions in which I asked the visiting dignitaries, diplomats, intelligence officials, generals and members of the U.S. intelligentsia if the U.S. would hand over Afghanistan to the Taliban,” Mr. Saleh said after Kabul’s fall to the Taliban. “The answer would be outright no, with nuances explained later but still implying no.”

As members of Mr. Ghani’s inner circle continued to cling to illusions, Afghan army and police field commanders drew a different conclusion: The end was nigh. Survival meant striking private deals with the Taliban and preparing for a rainy day meant selling off their units’ ammunition, food and fuel on the black market.

By May, the Taliban started taking one district after another, often without a fight, allowing government troops to go home unharmed and giving them pocket money for the road. Still, in accordance with verbal commitments given to Mr. Khalilzad, the insurgents refrained from seizing any of the country’s 34 provincial capitals. In Doha, Taliban negotiator Mohammad Nabi Omari, another former Guantánamo inmate who is affiliated with the Haqqani network, hashed out a transition proposal with a narrow circle of Afghan republic representatives.

Under the proposed deal, the Taliban’s supreme leader, Mr. Hibatullah, would become Afghanistan’s head of state but the country would turn into a constitutional monarchy of sorts, governed under the 1964 constitution promulgated by King Zahir Shah, with an elected parliament. Ms. Gailani, who was involved in this negotiation, joked that Mr. Hibatullah, who hadn’t been seen in public for years and widely presumed to be dead, was a perfect head of state. Her Taliban interlocutor assured her that Mr. Hibatullah was very much alive. Both sides agreed to keep the planned agreement secret.

“They were not easy. There were things on which they would absolutely not compromise upon. They would never accept the Islamic Republic of Afghanistan. They would never accept our constitution,” said Ms. Gailani. “But at least 60% of our values could be rescued. Our flag could be rescued.”

Mohammad Masoom Stanekzai, the lead Afghan government negotiator in Doha and a former defense minister and intelligence chief who regularly briefed Mr. Ghani on the talks, said he believed the plan presented by Mr. Omari was just an individual idea and not a solid proposal backed by the entire Taliban leadership.

In late June, Mr. Ghani flew to Washington in a last-ditch effort to persuade the U.S. of the need to keep providing support. Mr. Biden agreed to receive Mr. Ghani in the White House only if he came with Dr. Abdullah, then holding the title of head of Afghanistan’s High Council for National Reconciliation. “We’re going to stick with you. And we’re going to do our best to see to it you have the tools you need,” Mr. Biden promised in joint remarks.

The American president’s April withdrawal decision “has made everybody recalculate and reconsider,” Mr. Ghani chimed in. “The Afghan nation is in an 1861 moment, like President Lincoln, rallying to the defense of the republic. It’s a choice of values—the values of an exclusionary system or an inclusionary system.”

Ms. Gailani met Mr. Ghani in Washington during that trip and briefed him on proposals discussed with Mr. Omari and other Taliban negotiators. Mr. Ghani encouraged her to continue the talks, she said. “I thought, good, he decided to be the de Klerk of Afghanistan, not the Saddam or Gadhafi,” she recalled. “It was clear that this was the end, but at least it could have been a decent end. At least the institutions, the army, the police would not have collapsed.”

Yet, in following weeks, Mr. Ghani continued playing for time. “He lingered and lingered, which just made things more difficult,” Ms. Gailani said.

In July, a senior foreign envoy visited Mr. Ghani in Kabul. The Afghan president was defiant, boasting about the strength of government forces massed in the city and saying that the Taliban would suffer 50,000 casualties should they attempt to attack the capital. Still, he added that he instructed his bodyguards to give him a lethal injection should he face the risk of being captured by the Taliban, according to the envoy.

Gen. Kenneth McKenzie, the head of U.S. Central Command, flew to Kabul later that month to meet Mr. Ghani, publicly promising intensified airstrikes in support of Afghan forces. “Taliban victory is not inevitable,” he said at the time. In private, Gen. McKenzie told Mr. Ghani that Mr. Biden was still evaluating options for continuing to provide air support to Afghan forces from bases in the Persian Gulf after the withdrawal.

The Republic Collapses

In early August, the Taliban’s military commission chief, Mullah Abdul Qayyum Zakir, gathered military commanders in the insurgent stronghold of Aryub Zazi in the eastern Paktia province. The time to capture provincial capitals had come, Mr. Zakir announced, but the Taliban should take their time and not rush.

“It was decided that we should enter the cities cautiously, targeting the provinces that fall an easy prey,” said Hajji Qari Osman Ibrahimi, a member of the Taliban military commission who attended the meeting. “And we were told not to enter Kabul, because we had promised so to the Americans.”

As it turned out, almost all the cities were easy prey, and just a week later the Taliban were at the doorstep of the Afghan capital. Dr. Abdullah held another round of meetings in Doha and returned to Kabul to brief Mr. Ghani and other political leaders: A transitional arrangement that would save at least some of the Afghan republic’s institutions was still possible. The Taliban had a strong incentive to cooperate. The U.S. had assured the insurgents that such a transitional government would get diplomatic recognition and would have access to billions of dollars in Afghan central-bank reserves and continued foreign aid.

Dr. Abdullah, Mr. Karzai, Islamist warlord Gulbuddin Hekmatyar and other Afghan leaders planned to fly to Doha to strike such an agreement but needed Mr. Ghani’s commitment to resign first. Once again, the Afghan president stalled for time, haggling over the composition of the delegation and insisting that close aides such as Mr. Mohib participate. The delegation was tentatively scheduled to leave Aug. 16.

Amin Karim, a senior member of Mr. Hekmatyar’s party and a former adviser to Mr. Ghani, went to see the Afghan president in the palace that week.

“It’s game over,” he started the meeting, in English. Mr. Ghani, flustered, accused Mr. Karim of defeatism, saying that Kabul was safe and that tens of thousands of elite troops from all over the country were ready to protect the Afghan capital.

On Aug. 14, Mr. Wilson, the American envoy, also met with Mr. Ghani. By then, the major cities of Kandahar, Herat and Ghazni had fallen to the Taliban. He says he was struck by how calm the Afghan leader appeared. Reporters were invited to cover the meeting, which was unusual. Taliban commanders in the mountains around the city had no inkling that just hours later they would be in control of the Afghan capital.

“We were sure that provinces would fall without any resistance, but we weren’t sure about Kabul. Bluffing by the government had given us a sense that there would be a fight,” said Mohammad Salim Saad, a senior commander of the Haqqani network’s Badri force who oversaw insurgent operations within the capital. “We worried that a battle for Kabul would destroy the city.”

The morning of Aug. 15, some armed Taliban sympathizers started appearing in the city. On Washington’s request, the Taliban issued a statement in Doha that requested all Taliban units to stay away. Mr. Wilson ordered all remaining personnel to move from the U.S. Embassy compound in Kabul’s Green Zone to the airport, then held by the American military.

Remaining staff were told to leave their personal effects behind and were allowed just one suitcase. Mr. Wilson left his suits and shoes at the embassy, and packed the essentials, including a book that had just arrived via Amazon delivery. As he boarded the chopper to leave for the airport, the pilots told him that Mr. Ghani had been spotted fleeing Afghanistan by helicopter about 30 minutes earlier.

“He gave us no hint that he was leaving. Not a scintilla of a hint that he was going to leave the country,” Mr. Wilson recalled. Mr. Ghani, in a statement released weeks later from the United Arab Emirates, where he now resides, said his unexpected departure “was the only way to keep the guns silent and save Kabul.”

In Doha, senior Taliban representatives gathered on the 21st floor of Qatar’s foreign ministry for a meeting with the country’s special envoy who oversaw Afghan affairs, Mutlaq al Qahtani. In disbelief, they watched the news of Mr. Ghani’s escape. Would the U.S. military want to secure Kabul for two weeks, to enable an orderly transition, they asked.

Mr. Baradar, Mr. Khalilzad, Gen. McKenzie and other officials met in Doha that afternoon. “There was a sense of anarchy coming. Law and order was falling apart in Kabul,” Mr. Khalilzad recalled. Following Mr. Ghani’s escape, the rest of the Afghan republic’s ministers, including the minister of defense, also rushed to the airport to flee the country.

The Biden administration wasn’t interested in taking potentially open-ended responsibility for the besieged Afghan capital and its five million residents. “It’s not my job. My job is to safely withdraw my forces,” Gen. McKenzie replied to the Taliban proposal, according to Mr. Khalilzad. “If you attack, we’ll defend ourselves.”

By 8 p.m., Taliban units, mostly those belonging to the Haqqani network, started entering the city, reinforcing the first echelon of clandestine operatives who had seized strategic locations.

Instead of a negotiated transfer of power with international recognition that had been discussed with the U.S., the Taliban found themselves running a government with empty coffers, subjected to American sanctions and denied a United Nations seat.

Mr. Baradar, widely expected to become the Taliban’s new head of government, was marginalized as one of three deputy prime ministers, and later disappeared from view for weeks. His verbal promises to American and other international negotiators, such as a commitment to ensure girls’ education, were no longer binding for Afghanistan’s new regime.

Instead, the Haqqanis and the southern military commanders under Mullah Omar’s son Mullah Mohammad Yaqoob emerged as the factions with real authority in Kabul.

A newly published Taliban calendar, for the Islamic year that began in August 2021, no longer carried Mr. Hibatullah’s promise of not seeking a monopoly on power. Instead, it pledged to enforce a “pure Islamic system.” A pile of wrecked Humvees left behind and a fleet of Chinooks flying away with tattered American flags illustrated the message.

—Gordon Lubold contributed to this article.

Write to Yaroslav Trofimov at [email protected] and Jessica Donati at [email protected]