NEW YORK (AP) — A federal judge made clear Wednesday that the NFL’s four game suspension of Tom Brady over “Deflategate” is in jeopardy as the star New England quarterback returned to practice.

U.S. District Judge Richard M. Berman, who’s been asked by the NFL Players Association to void the suspension, warned a league lawyer during oral arguments in the scandal over underinflated footballs that there was precedent for judges to toss out penalties issued by arbitrators.

Berman continued to push for a settlement in the dispute — a potential result he called “rational and logical.” But throughout the hearing, he also cited several weaknesses in the way the NFL handled the controversy that could become the basis for handing a victory to Brady and his union.

After the hearing, Berman met behind closed doors with both sides for more than an hour before the lawyers left court, saying the judge asked them not to discuss the negotiations publicly. If there is no deal, the Manhattan judge has said he hopes to rule by Sept. 4, six days before the Patriots host the Pittsburgh Steelers in the NFL’s season-opening game.

Neither Brady nor NFL Commissioner Roger Goodell was in court Wednesday. Brady returned to his team after participating in negotiations along with Goodell and lawyers on both sides a day earlier.

Berman ordered Brady and Goodell to return to court if they can’t settle, scheduling a tentative Aug. 31 hearing.

The league announced in May that it was suspending Brady over allegations he conspired with two Patriots equipment employees to deflate footballs below what league rules allow, to give him a competitive edge in New England’s victory over the Indianapolis Colts in January’s AFC championship game. Goodell, who by contract with the players’ union can act as an arbitrator for labor disputes, upheld the suspension, touching off the legal battle.

During more than two hours of arguments by attorneys, the judge noted other arbitration decisions have been rejected when a key witness was not allowed to testify as he asked why NFL Executive Vice President Jeff Pash — who worked on the NFL investigation — could not be questioned by union lawyers during the suspension’s appeal.

Arbitration proceedings, while more relaxed than court proceedings, are still required to follow due process rules to ensure fairness, Berman said.

“You have to allow someone to make their case by calling witnesses,” he said.

Berman also suggested that the league’s finding that Brady was generally aware that game balls were being deflated was too vague, noting that any reference to the Jan. 18 game against the Colts was “conspicuously absent” in a report on an NFL investigation that the league used as a basis for the suspension.

Finally, Berman said he could not understand how the commissioner opted to keep a four-game suspension over a fine or a lesser penalty seen in other cases of equipment tampering. In one exchange, he questioned Goodell’s defense of the Brady punishment on the grounds that it was comparable to penalties on players caught using performance enhancing drugs.

“How is that equal to steroid use?” he asked of the deflation allegations.

“They both go to the integrity of the game,” responded NFL lawyer Daniel Nash.

“Well, everything goes to the integrity of the game,” the judge shot back.

It was the second week in a row the judge seemed to lean harder on the NFL in open court, though he again cautioned that he had not yet made up his mind which side would win.