How does earth based technology interact with alien technology

There is a deleted scene that explains some of this, added to the Extended Edition DVD, mentioned in this article (second article in the list).

Basically, remember when at the start of the movie, Jeff Goldblum managed to decode the alien mothership countdown? In a deleted scene, Goldblum mentions that he has deciphered their programming language and that the 50 year old Roswell ship is using the same programming language and operating system. So it's not a stretch to assume that he managed to write a rudimentary virus.

As for the transfer, that's handled by the communication between the mothership and the infiltrator craft. That was still intact because the technology was still the same. The virus was not written on the mac and then transfered to the mothership. The mac interfaced with the Roswell ship, developed the virus on there and could control the ship's onboard computer, but it didn't actually communicate directly with the mothership.

There's a bit more info in the film's official novelisation, co-authored by the film's producer, Dean Devlin.

How did they connect? Via wires, of course

When David is in the Hangar, the laptop is connected directly into the shield unit on the shuttle. He tranfers commands to it via a cable which means that no wireless connection was needed...

David walked over to a rolling tool shelf where he’d set up his laptop computer. It was connected to a cable that ran through the shield, into the cockpit of the alien craft, and plugged into the shield receiving unit he’d repaired earlier that day.

Later, when they're on the ship they connect to the mothership via the fin on the top of the smaller craft...

“Dr. Okun showed me that the long fin like structure on top of the attacker is full of terminal wiring. He hypothesized that whatever type of computer link they run, the fin is the connector. When one of these attackers docks inside the larger ships, some type of connection is established through the fin.”

...and transfer the virus file from the laptop to the shuttle to the mothership via wired connections at every stage...

The moment the clamps locked them in place, the movements on the screen, which showed the status of the protective shield, reversed their direction. That told David that they were connected to the source. He switched over to another screen, which flashed the words “Negotiating with Host.” He held his breath as the signal analyzer program sorted through the billions of possibilities. Then, much sooner than he expected, the machine beeped and displayed a new message, “Connecting to Host.”

How are they able to hack the alien ship? - because the aliens program in binary

As to how they were able to communicate with the alien ship, this is explained as being a function of basic h4x0r skillz. The aliens program their computers in binary and that leaves them exposed to being infiltrated:

It only took the pair of brainiacs a few minutes to solve their first problem. They tore the green screen loose and delicately attached a pair of alligator clasps to the sinewy wiring on the back.

Although the machine had been built thousands of years before in another part of the universe, the data feeding into it was arranged in a binary system, a continuous string of ones and zeros, or whatever the alien equivalent was. Neither Okun nor David cared at that point’ just as long as the little laptop could read the sequence.