np bro idk about silverman modern calc ngl. never tried it or heard of it mb. have fun vro also i literally made an account to answer this LOL i got banned initially because i got flagged for making an account and instantly posting a longass message

for linear algebra, the gold standard texts are hoffman-kunze and friedberg-insel-spence (xd insel). be wary of suggestions of sheldon axler's linear algebra done right. also the title is turbo ego which is a crazy red flag. it's basically only good if you are ready for some abstraction and already have semi-rigorous understanding of linear algebra, which is, frankly, a really weird place to be in educationally. if you can handle a lot of abstraction though, it's fine. you can follow this with steven roman's advanced linear algebra. i don't recommend it as a first course, though.
if you're a turbochad you can open with artin's algebra book because it has like two or three chapters on linear algebra, but that's probably not comprehensive and honestly kind of dicey. tl;dr go hoffman-kunze or friedberg-insel-spence and profit

as a follow-up, frankly, i don't recommend rudin altogether, unless you have someone to teach it to you. it's not even a matter of "oh you're not skilled enough xd" but rather "this book fucking sucks shit to read; even if you have a phd it's boring as fuck" because it's basically just a list of theorems and proofs with zero motivation, examples, pictures, etc. if you want further analysis reading, read pugh or zorich or something. just don't read rudin. you can take exercises from it tho fs.
further reading in algebra is usually dummit and foote abstract algebra. munkres for topology. ahlfors for complex analysis. enjoy.

use spivak calculus. i think cummings is very basic and nice, but i haven't read it myself so i can't possibly recommend it. from what i've heard though, it's a very nice introduction at absurdly low price because jay cummings specifically wrote it to be accessible in both intellectual and economic ways.
in terms of spivak, what i will say is that it is a very rigorous and complete introduction to singlevariable analysis. it may honestly be that all you need is spivak. he writes very lyrically, as opposed to rudin's outrageously dry manner. it's almost like comparing a hearty stew to dryass fucking sandy chicken breast. the chicken breast is surely nutritional, but most of the time, you can't eat it without something else (read: a professor). basically, spivak is a really nice and smooth introduction to from calc bc to end of singlevariable analysis with lots of motivation, examples, diagrams, etc. everything you need to self-study. the exercises are also as hard as or harder than rudin's, so if you are the goat, you will be able to do 30% of them. unironically. it is rare that even a very talented student can do more than 50% of them with no experience and no help.